retek

A Kubernetes operator for Temporal workers

Scale Temporal workers by the work they are actually doing.

retek replaces KEDA and Prometheus for Temporal worker fleets. It reads the task-queue backlog and the slots busy inside every worker pod, so a busy fleet never looks idle — and computes one replica count that is safe to write.

demand = Σ backlog of declared queues + Σ busy activity slots on ready pods

Get started How it decides Migrate from KEDA

The problem it solves

Backlog-only autoscalers have a blind spot the moment work is picked up. A queue can empty in one poll cycle because every idle worker grabs a task at once — so a fully busy fleet reads as zero backlog and looks idle. KEDA then holds or scales down the very Deployment that is saturated, and the next burst waits for pods that no longer exist.

retek counts what KEDA cannot see: how many activity slots inside each worker are in use, reported per pod by an agent embedded in the worker process. Demand is the backlog plus those busy slots, so work in flight counts as demand, not as silence.

A taste of the API

scaler.yaml
apiVersion: retek.colossyan.com/v1alpha1
kind: TemporalConnection
metadata:
  name: temporal-main
spec:
  hostPort: temporal-frontend.temporal.svc.cluster.local:7233
---
apiVersion: retek.colossyan.com/v1alpha1
kind: TaskQueueScaler
metadata:
  name: render-worker
  namespace: rendering
spec:
  mode: Active
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: render-worker
  temporal:
    connectionRef:
      name: temporal-main
    namespace: default
  taskQueues:
    - name: render-activities
      type: activity
  targetValue: 4            # one replica per 4 units of demand
  minReplicas: 0
  maxReplicas: 40
  pollingIntervalSeconds: 10
  workerAgent:
    enabled: true           # read busy slots from the pods
    port: 9090
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 120

One CRD to scale a Deployment, one CRD to reach Temporal. Everything else — HPA-style behaviour, cooldowns, safety gates — is fields on the scaler.

Why retek

Accurate demand

Backlog plus busy slots, measured per pod. Work that has already been taken off the queue still counts, so a saturated fleet scales up instead of down.

HPA-identical behaviour

Stabilization windows, scale policies, selectPolicy and tolerance are ported from the HPA, with KEDA-style activation and cooldown on top. No new semantics to learn.

Health you can act on

Pods that cannot be read are never assumed idle — they freeze scale-down. Conflicts, coverage gaps and Temporal outages are conditions and metrics, not surprises in a log.

Status: what is shipped, what is planned

shipped

  • Active and Shadow autoscaling, scale to zero, HPA-style behaviour
  • The Python worker agent, contract v2 (v1 still accepted)
  • Conflict detection against HPA and KEDA
  • The retek convert migration CLI
  • Paused scalers, degraded mode, Temporal fallback

planned

  • Drain on scale-down — the API field exists but is rejected until it ships
  • Karpenter disruption protection, stuck-activity detection
  • Lifecycle metrics, the Go agent, OpenTelemetry export

See Limitations for the precise list — nothing is advertised here that the operator does not do today.