Production operator

scribase-operator run is the production worker loop. Each cycle claims an operation in PostgreSQL, commits the lease, performs at most one real Neon or Kubernetes action, then opens a new fenced checkpoint transaction. An idle worker polls at a bounded interval. Control-plane failures use bounded exponential backoff and terminate the process after a configurable consecutive failure threshold so the orchestrator can surface an unhealthy deployment. Provider failures do not crash the worker because the reconciler has already persisted their retry or terminal checkpoint.

scribase-operator reconcile-once executes the same production path for a single cycle and remains useful for Kubernetes Jobs and diagnostics.

Before claiming work, each invocation atomically schedules at most one expired preview in the selected organization. Candidate selection uses FOR UPDATE SKIP LOCKED; the same transaction creates the delete operation, cancels superseded work, advances desired state, and records the expiry audit event. A second invocation cannot schedule the preview again once its phase is deleting.

Trusted gateway or telemetry collectors renew an active preview with:

scribase-operator observe-activity acme billing pr-42

The optional fourth argument is the observed Unix epoch in seconds, which lets a collector forward event time instead of process time. Collectors should coalesce events (for example, one renewal per active preview every five minutes) rather than emit one database write per request. The transaction accepts only a newer event and rejects timestamps more than five minutes ahead of PostgreSQL time. Missing, stale, non-preview, deleting, and deleted targets are safe no-ops. Each accepted renewal is attributed to system:gateway-activity in the immutable audit log. This command needs only the operator database URL and psql; it does not initialize Neon or Kubernetes clients.

Required configuration:

Variable	Purpose
SCRIBASE_WORKER_ID	Stable worker identity used in the lease fence
SCRIBASE_OPERATOR_DATABASE_URL	TLS URL for scribase_operator and provider resource mappings
SCRIBASE_NEON_API_KEY	Neon v2 API credential
SCRIBASE_PUBLIC_DOMAIN	Parent DNS name for environment gateways
SCRIBASE_STORAGE_ENDPOINT	S3-compatible Storage endpoint
SCRIBASE_STORAGE_BUCKET	S3-compatible Storage bucket
SCRIBASE_STORAGE_REGION	S3-compatible Storage region
SCRIBASE_EXTERNAL_SECRET_STORE	External Secrets Operator store name
SCRIBASE_EXTERNAL_SECRET_PREFIX	Remote path prefix owned by Scribase environments
SCRIBASE_RUNTIME_IMAGE_GATEWAY	Envoy image pinned as repository@sha256:<64 hex>
SCRIBASE_RUNTIME_IMAGE_AUTH	Supabase Auth image pinned by digest
SCRIBASE_RUNTIME_IMAGE_REST	PostgREST image pinned by digest
SCRIBASE_RUNTIME_IMAGE_REALTIME	Supabase Realtime image pinned by digest
SCRIBASE_RUNTIME_IMAGE_STORAGE	Supabase Storage image pinned by digest

One operator serves every organization. It discovers the organizations with claimable work (a due operation or an expired preview) through the identifier-only functions of migration 0021_operator_organization_queue.sql, then claims each tenant's queue under that tenant's forced-RLS context. It serves organizations round-robin: at most SCRIBASE_OPERATOR_ORGANIZATION_BATCH (default 25) organizations per turn, in id order behind a cursor that wraps, and at most SCRIBASE_OPERATOR_OPERATIONS_PER_TURN (default 4) consecutive actions per organization before moving on. The tenant schema sweep on start and migrate-tenants run for every organization, and engine moves run for every organization that has one queued. To run a dedicated operator for a single tenant instead, set SCRIBASE_OPERATOR_SCOPE=organization and SCRIBASE_ORGANIZATION_ID.

The database URL must explicitly contain sslmode=require. PostgreSQL 16+ psql, curl, jq, and kubectl must be installed in the operator image. External Secrets Operator and the configured SecretStore or ClusterSecretStore must already exist. Scribase server-side-applies two environment-scoped external-secrets.io/v1 resources, waits for their Ready conditions, then deploys workloads whose required secretKeyRef entries provide the final key-level readiness check. The operator service account has no Secret read permission, and Scribase never places secret values in desired-state records or Kubernetes manifests. Every runtime image is required at startup and must use an immutable SHA-256 digest. Mutable tags from versions.lock.toml identify qualification inputs but are not accepted by the production operator; the release pipeline must resolve and record their registry digests before deployment.

For organization acme, project billing, and environment pr-42, the two remote objects are:

<prefix>/acme/billing/pr-42/runtime
<prefix>/acme/billing/pr-42/database

The runtime object must expose ANON_KEY, DB_ENC_KEY, JWT_SECRET, S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY, SECRET_KEY_BASE, and SERVICE_ROLE_KEY. The database object must expose AUTH_DATABASE_URL, REALTIME_DB_HOST, REALTIME_DB_NAME, REALTIME_DB_PASSWORD, REALTIME_DB_PORT, REALTIME_DB_USER, REST_DATABASE_URL, and STORAGE_DATABASE_URL. The secret-manager provisioning workflow must set the matching database role passwords before it publishes these values.

Optional configuration:

Variable	Default
SCRIBASE_LEASE_SECONDS	60
SCRIBASE_DATABASE_MAX_CONNECTIONS	10
SCRIBASE_NEON_API_BASE	https://console.neon.tech/api/v2
SCRIBASE_NEON_DATABASE	neondb
SCRIBASE_NEON_ADMIN_ROLE	neondb_owner
SCRIBASE_GATEWAY_NAMESPACE	scribase-system
SCRIBASE_GATEWAY_NAME	scribase-gateway
SCRIBASE_KUBERNETES_TIMEOUT_SECONDS	30
SCRIBASE_KUBERNETES_CONTEXT	current context
SCRIBASE_EXTERNAL_SECRET_STORE_KIND	ClusterSecretStore; may be SecretStore
SCRIBASE_EXTERNAL_SECRET_REFRESH_SECONDS	300 (accepted range 10–86400)
SCRIBASE_POLL_INTERVAL_MILLISECONDS	1000 (accepted range 50–60000)
SCRIBASE_MAX_CONSECUTIVE_ERRORS	20 (accepted range 1–10000)
SCRIBASE_ERROR_BACKOFF_MAX_SECONDS	30 (accepted range 1–300)
SCRIBASE_RECLAIM_INTERVAL_SECONDS	300 (accepted range 10–86400): how often idle cycles delete the engine projects (for example the managed Neon project) of deleted Scribase projects, from scribase.released_provider_resources (discovered through migration 0022_operator_released_resources.sql). Failed deletions stay queued for the next pass
SCRIBASE_PSQL_BINARY	psql
SCRIBASE_KUBECTL_BINARY	kubectl

The explicit self-check command is isolated verification only. It is the only operator command that uses in-memory fake providers and must not be used as a deployment entry point.