Skip to main content
op-data is a hybrid control plane / data plane system. The control plane orchestrates; the data plane touches your data. The boundary between them is the most important contract in the product.

​
The short version

  • op-data never sees raw source rows or canonical output — except for preview queries, which are bounded and user-triggered (see below).
  • op-data never stores source credentials. Connector passwords, OAuth refresh tokens, and DSNs live in your agent’s local Postgres. The control plane stores non-secret metadata only.
  • op-data never opens inbound connections to your environment. The agent connects outbound to op-data over TLS on port 443.
  • op-data cannot execute arbitrary code on your agent. Workflow definitions are shipped in the agent image; upgrading means pulling a new image tag, not accepting new code at runtime.

​
The data boundary

Data classWhere it livesNotes
Source credentials (DB passwords, tokens)Customer envAgent-local Postgres, Connection.payload JSON.
Raw source rowsCustomer envAgent reads, transforms, and writes without round-tripping through the control plane.
Canonical model outputCustomer envWritten to your sink (S3, warehouse, Postgres).
Connection metadata (host, db name, SSL mode)op-data control planeConvex connections.safeMetadata.
Pipeline definitions, schedules, model configsop-data control planeThe thing you manage in the UI.
Run metadata (row counts, byte counts, durations, schema fingerprints)op-data control planeReturned by Temporal activities. No row content.
Preview query resultsop-data control planeException — see below.
Every Temporal activity return type is a metadata shape — counts, durations, hashes, opaque references. Orchestration inputs carry identifiers (orgId, connectionId, modelId), not credentials or embedded data.

​
The preview-query exception

previewQueryWorkflow is the one path that returns real rows to the control plane. It exists so the UI can render a tabular preview when you are authoring a model. It is bounded by construction:
  • User-triggered only. It does not run on a schedule.
  • SELECT / WITH only. The agent rejects any other statement type.
  • Single statement. Semicolon chains are rejected.
  • Hard row cap of 100. The agent wraps the query in select * from (<sql>) as preview_query limit 100.
  • 10-second timeout.
  • Per-cell text truncation at 1024 characters.
  • Value sanitization before the rows leave the agent.
Canonical sync does not use this path. Sync flows return only metadata — row counts, byte counts, and schema fingerprints — never source data.

​
Where credentials live

Connector secrets are stored in the agent’s local Postgres database (OP_AGENT_DATABASE_URL). The Connection table has:
  • id — stable connection identifier, also stored in the control plane.
  • type — connector kind (e.g. postgres).
  • payload — opaque JSON credential blob.
  • workspaceId — tenant scope.
  • name — display label.
Our control plane’s connections table (in Convex) stores only the non-secret metadata needed to render the UI: name, type, status, and safeMetadata (host, database, SSL mode). It never stores passwords, refresh tokens, or DSNs. Back up the agent database. If you rebuild it without a restore, the Convex metadata remains but the secret payloads are lost, and every connection must be re-saved from the UI. External secret managers (AWS Secrets Manager, GCP Secret Manager, Vault) are optional integrations. The default deployment does not require any of them.

​
Authentication

Users. Human auth is handled by WorkOS AuthKit. Every user-facing Convex query and mutation goes through requireAuth(ctx), which resolves { orgId, userId } from a signed session. Cross-tenant access is prevented at the query layer. Agents. Each agent deployment has its own WorkOS Connect M2M client (client ID + secret) provisioned via the UI. The agent exchanges the credentials for a short-lived token and attaches it to every Convex call. You can revoke an agent’s access instantly by clicking Revoke on its deployment page — this deletes the WorkOS client secret. Temporal. The agent connects outbound to op-data’s Temporal frontend and polls a tenant-scoped task queue (tenant-{orgId}). The queue name is scoped by tenant so a misconfigured agent cannot pick up another tenant’s work. mTLS is supported via OP_TEMPORAL_TLS_CERT / OP_TEMPORAL_TLS_KEY.

​
Tenancy

  • Deployment boundary. One agent per tenant, running in the tenant’s own environment.
  • Task-queue boundary. Workflows are routed by tenant-{orgId} task queue; workers only pick up work for their own tenant.
  • Auth boundary. requireAuth(ctx) enforces orgId on every user-facing operation in the control plane.

​
Upgrading without remote code execution

Control-plane upgrades do not push code to your agent. The agent runs the workflow definitions baked into its image; newer workflow types become available only when you pull a new image tag. In-flight executions replay safely across upgrades — workflow inputs are append-only, and logic changes use Temporal’s patched() to avoid non-determinism.

​
Reporting a security issue

Email security@op-data.com. PGP key and response SLA are published at op-data.com/security.