The InfraOrcinus Native State Backend: Atomic Locking and Encrypted State

By Chirag Panchal · Founder & Lead Architect · · 8 min read

How InfraOrcinus implements its own native HTTP remote state backend. An architectural deep-dive into atomic PostgreSQL locks, AES-256-GCM encryption of state at rest with optional Vault Transit, and what the backend does and does not yet provide.

Development Status & Architectural Evolution: InfraOrcinus is in active development. Architecture specifications, internal interfaces, and operational protocols described herein reflect our current engineering design and are continuously refined to make InfraOrcinus robust, resilient, and enterprise-grade.

In Infrastructure as Code, the state file is the crown jewels of your cloud estate. It holds the authoritative mapping between declarative code definitions and physical cloud resources across AWS, Azure, and Google Cloud.

However, state files present two severe challenges:

  1. Plaintext Secrets: Terraform state files inherently store sensitive resource outputs in plaintext - including database master passwords, private TLS certificates, and API tokens.
  2. Concurrency Collisions: If two automation pipelines or engineers execute plans against the same state without mutual exclusion, resources can be duplicated, corrupted, or deleted out of order.

To solve this, InfraOrcinus provides its own Native HTTP Remote State Backend.

1. Architecture of the Native State Backend

Instead of requiring teams to configure separate external cloud storage buckets (such as Amazon S3 with DynamoDB tables, or Azure Blob Storage with Blob leases), InfraOrcinus acts as an authenticated HTTP state backend compatible with both Terraform and OpenTofu:

terraform {
  backend "http" {
    address        = "https://control-plane.example.com/api/v1/state/stack-prod-vpc"
    lock_address   = "https://control-plane.example.com/api/v1/state/stack-prod-vpc/lock"
    unlock_address = "https://control-plane.example.com/api/v1/state/stack-prod-vpc/unlock"
  }
}
InfraOrcinus Native State Backend Protocol
Figure 1: The InfraOrcinus state lifecycle. Runner requests acquire an atomic database lock, the state is encrypted at the application layer with AES-256-GCM, and it is stored as one document per stack.

Authentication to the Backend

Today the native HTTP state endpoints are used by the platform's runners. The runner authenticates to the control plane with its own credential, so the backend block in a run needs no secret. People and external tooling cannot reach the backend with an API token yet. Commands such as terraform state pull against InfraOrcinus are therefore not available today. Direct user access is on the roadmap, and it will use the same organization and project scoping as the rest of the product.

Scope: Which Engines Use This Backend

The HTTP backend protocol is implemented by Terraform and OpenTofu, and that is what this article describes. Pulumi has its own state model and backend selection, covered in the multi-engine article. Stacks can alternatively use a remote cloud backend (Azure Blob, Amazon S3, or Google Cloud Storage) when state must live in your own account. The controls below apply to the native backend.

2. Distributed Locking with Atomic Database Writes

State locking is not an optional feature; it is an invariant requirement for infrastructure safety.

The lock endpoints are implemented as one atomic, conditional database write. A lock is taken only when the stored lock ID is empty. Concurrent requests race on that single write, and exactly one wins.

  • Lock metadata. The lock record stores the lock ID and the lock information Terraform sends (operation, who, and when). A second client receives HTTP 409 Conflict with the message that the state is locked by a different client, and Terraform reports the holder.
  • Proof of ownership on unlock. Releasing a lock requires the lock ID that acquired it. A mismatched unlock is refused, and the lock stays in place, so one run cannot release another run's lock.
  • Lease expiry is not built. A lock held by a dead runner is not released automatically. Job claims do have leases and a reaper for stuck jobs, but state locks do not expire yet. Clear a stranded lock using the procedure below.

Stranded Locks Are an Operator Action Today

The service has a force-unlock operation that releases a lock regardless of its owner. It is not yet exposed as an API route or a UI action. Until it is, handle a stranded lock like this:

  1. Check the run list and confirm that no run for the stack is still in flight. Force-unlocking during an apply is how state corruption happens.
  2. Release the lock through your maintenance procedure, and record the reason in your change ticket.

The intended design is administrator-only force-unlock, with an audit record naming the actor, the stack and the lock released. That is not shipped yet.

3. Encryption at Rest

State payloads are encrypted before they are written to the database, using AES-256-GCM, which is authenticated encryption. Encrypted values carry a version prefix (enc:v1:), so a stored value is never silently read as plaintext.

  • Key source. The key comes from configuration. A key of the wrong length stops the server from starting, and the built-in development secrets are refused in production.
  • Vault Transit (optional). When configured, Vault Transit can serve as the encryption provider in place of a local key.
  • Database-side pgcrypto (optional). The platform ships a migration that encrypts secret sub-fields inside the database. It is a second layer for secrets, not a per-state mechanism.
  • Per-stack data keys and cloud KMS wrapping are not built. The design calls for a data key per stack, wrapped by a key-encryption key in your KMS. Today all state is protected by the platform key, so the key's custody is the main control.

Key Management and Rotation

  • Keep the platform key in your secret manager, and back it up separately from the database. A database backup without its key restores ciphertext nobody can read, which is the point.
  • Rotating the platform key is not an online operation yet. Plan rotation as a maintenance window: re-encrypt, verify, then retire the old key.

Who Can See Decrypted State?

Decryption happens inside the control plane. Sensitive values are masked on the server before any state-derived view, such as the resource graph, reaches the browser, and credentials are never returned by the API once stored. People cannot read raw state through the API today, so the main exposure is the UI views. Restrict those by project.

Cross-stack reads, for example one stack consuming another's outputs with terraform_remote_state, expose the other stack's outputs. Grant them deliberately, and keep sensitive outputs out of them.

4. Current Storage: One Document per Stack

Each stack's state is stored as one document and is replaced on every write. The backend does not keep older versions, so there is no built-in state history, and no rollback to a previous version, yet.

  • Where history lives today: the run records (who ran what, and when), the approval records, and the audit trail. Use these to reason about changes. They are not a substitute for a state backup.
  • Backups: back up the database. Because state is encrypted, a backup is only useful together with the platform key (section 3).
  • Planned: versioned state snapshots, a checksum per version, and rollback to a verified version.

5. Large State, Migration In, and Exit Out

Large State Documents

State for a big estate can reach several megabytes of nested JSON. PostgreSQL stores such payloads out-of-line with automatic compression (TOAST), so there is no per-document size ceiling of the kind imposed by some document databases. Split very large stacks along natural boundaries, which also reduces blast radius and lock contention.

Migrating State In

Moving an existing stack onto the native backend is a standard Terraform operation, and init -migrate-state is the usual command. Today the native backend is used by runners only, so migrating state in is not yet a supported workflow. For stacks you are migrating now, use a remote backend you already run (Amazon S3, Azure Blob or Google Cloud Storage), and revisit the native backend when direct access ships.

Exit Path

Leaving the native backend is not a one-command operation yet. Read the state out through your maintenance procedure, and write it to your own backend with terraform state push. A direct export is on the roadmap. A platform that holds your state should make leaving easy, and the export path is part of what we are building.

Summary

The native state backend keeps locking and encryption in the control plane, so infrastructure secrets stay protected across deployments. The current release is runner-only, locks are atomic and owner-checked, and state is encrypted with one platform key. Direct access, lock expiry, versioned history and rollback are the next steps.