ADR 0014: Exo Secure loads secrets from 1Password at boot; no secrets in platform env varsCopy Anchor Link

• Status: proposed (draft PR; owner decision pending on the open items below)
• Date: 2026-10-21
• Decided by: Samir (owner), standing rule set 2026-10-21
• Relates to: ADR 0004 (Exo ID issuer and JWKS), ADR 0005 (EXO Verify), roadmap item "Exo Secure"

ContextCopy Anchor Link

On 2026-10-21 Railway disclosed that an unauthorized party accessed systems that store customer environment variables, obtained the decrypted contents of some of them, and appears to have been hunting for AI provider keys; Railway recommends rolling every variable and paused all deployments (Railway disclosure).

Every EXO secret that lives in a platform env store (Railway, Vercel, Convex) is exposed to a breach of that platform, and an env var also leaks through crash reports, child processes, env dumps and build logs. Separately, staging.exo.now proves the gateway to five upstream Apps (Exo, Moonshot, Hadron, Observatory, Universe) with one shared STAGING_GATEWAY_SECRET in the x-exo-gateway-secret header, so one leaked App env lets anyone impersonate the gateway to all of them (security review item M1).

Standing rule from the owner: Exo Secure must never store secrets as platform env vars. The only exception is a single bootstrap token per app.

DecisionCopy Anchor Link

1. One read-only 1Password service account per app environmentCopy Anchor Link

• Each app environment gets its own vault and one read-only service account scoped to only that vault: Exo Staging and Exo Production for EXO itself; every managed app follows the same pattern (Moonshot Staging, …).
• The platform holds only OP_SERVICE_ACCOUNT_TOKEN (the bootstrap token) plus non-secret config: EXO_SECURE_SOURCE=1password and EXO_SECURE_VAULT=<vault name>.
• Secrets are referenced as op://<vault>/<item>/<field>. References are not secret; they live in code (src/lib/exo-secure/manifest.ts), so the list of what an app needs is reviewed like code.

2. Resolve at boot into memory, never back into process.envCopy Anchor Link

• src/instrumentation.ts register() calls loadSecrets(EXO_SECRET_MANIFEST) once, before the server takes traffic. It uses the official @1password/sdk (MIT, v0.5.0) with one batched secrets.resolveAll call.
• Values go into an in-memory store on globalThis (a registered symbol, so the Next proxy bundle and route bundles share it). Server code reads them with secret(name) / requireSecret(name). Nothing is written to process.env, so env dumps and child processes never carry them.
• Fail closed in production. With EXO_SECURE_SOURCE=1password in production, a missing token, an unreachable vault, or any missing required secret throws in register(): the new deploy never becomes healthy and the previous one stays live. Staging and development log a warning instead.
• After loading, the server warns (by name, never value) about any managed secret that is still also set as a platform variable, so leftovers get deleted.
• Migration mode. EXO_SECURE_SOURCE defaults to env, which reads process.env exactly as today and warns in staging and production. Merging this ADR therefore changes no behaviour until an environment is switched to 1password.
• Local development uses either op run --env-file=.env.op -- pnpm dev (resolved values exist only in that child process; default env source) or EXO_SECURE_SOURCE=file with a gitignored .env.secrets.local (the .env.* rule already ignores it). file is refused in production.

3. Bird and other Convex-only secrets: Convex env as the interim homeCopy Anchor Link

Bird SMS already runs only in Convex (convex/lib/bird.ts, convex/communications.ts sendSms internal mutation and deliverSms internal action). No Next code reads BIRD_* or calls the Bird API; tests/int/exo-secure-boundaries.int.spec.ts now fails if that ever changes. Both functions are registered internal, so only Convex code (phone sign-in in convex/phoneAuth.ts, or a deploy-key caller) can trigger a send; they need no entry in the public-write allowlist of tests/int/convex-auth-guards.int.spec.ts. When Bird email arrives it follows the same shape.

Convex env is also a platform env store. Options:

Why A now: the breach was Railway's, and Convex is a separate vendor and blast radius; Convex functions run inside Convex, so these secrets never touch Railway or the Next server; and B still needs an OP_SERVICE_ACCOUNT_TOKEN in Convex env (the bootstrap exception moves, it does not disappear), adds a 1Password round trip to Node-action cold starts, cannot serve the default (V8) runtime that most EXO functions use, and spends service-account rate limit (Teams: 1,000 reads per hour per token; Business: 10,000; rate limits). Revisit B for EXO_ID_PRIVATE_KEY and AUTH_PRIVATE_KEY when Convex supports a secret-manager integration or if Convex's posture changes. Run the new platform-env check against npx convex env list output with an explicit allowlist to keep that store reviewed.

4. Per-app Exo Key gateway tokens replace STAGING_GATEWAY_SECRET (M1)Copy Anchor Link

• The gateway asks Exo Key for a token per upstream App: POST <exo key site>/exo-key/v1/gateway-token {"app": "<slug>"} with Authorization: Bearer <EXO_GATEWAY_CLIENT_SECRET> (in the Exo Staging vault). Convex stores only EXO_GATEWAY_CLIENT_SECRET_SHA256, which is not a secret. The route answers 404 until that hash is set, so it ships dark.
• The token is an RS256 JWT signed by the existing Exo ID issuer key (convex/lib/gatewayToken.ts, same kid as access tokens): iss = the Exo ID issuer, aud = exo-gateway:<app>, sub = exo-gateway, token_use = gateway, random jti, 5-minute lifetime. EXO caches one per App until a minute before expiry and forwards it in x-exo-gateway-token.
• Each App verifies against the public JWKS (<issuer origin>/exo-key/.well-known/jwks.json) with verifyGatewayToken (src/platform/delivery/gatewayTokens.ts). A token minted for one App fails at every other App, and an ordinary Exo ID access token is never accepted (no token_use). Apps hold no gateway secret at all.
• Flag EXO_GATEWAY_TOKENS on the EXO service: off (default; shared header only, today's behaviour), dual (both headers), only (token only; a missing token fails closed with 503). Caller-supplied copies of either header are always stripped.

Per-app migration (each step is reversible by flipping the flag back):

1. Generate the gateway client secret (48+ random bytes) into Exo Staging/staging-gateway/exo-key-client-secret; set its SHA-256 (base64url) as EXO_GATEWAY_CLIENT_SECRET_SHA256 in the staging Convex deployment.
2. Move the current shared value into Exo Staging/staging-gateway/legacy-shared-secret, switch the EXO staging service to EXO_SECURE_SOURCE=1password, delete STAGING_GATEWAY_SECRET from it.
3. Set EXO_GATEWAY_TOKENS=dual on EXO staging. Verify with a request through staging.exo.now/{app} that both headers reach the upstream.
4. For each App (Moonshot, Hadron, Observatory, Universe; then Construct, Rampart, Cosmograph, Vector, Dropship as they adopt the gateway): ship verifyGatewayRequest with acceptLegacy: true, deploy, confirm requests verify as token; then set acceptLegacy: false and delete STAGING_GATEWAY_SECRET from that App's service.
5. When every App is on tokens only, set EXO_GATEWAY_TOKENS=only, delete the legacy item from the vault, and remove the legacy code path.

5. EXO Verify enforces itCopy Anchor Link

exo-verify security gains a built-in platform-env check (packages/exo-verify/lib/level1/platform-env.mjs, plain Node, no git, no network). Given a variables export (railway variables --json, a [{name,value}] list, or dotenv; file path or stdin) it fails on values that look like secrets: known prefixes (sk-, sk_live_, ghp_, github_pat_, xox[bpas]-, re_, bk_, AKIA, ops_ other than OP_SERVICE_ACCOUNT_TOKEN, Stripe whsec_, Google AIza, Convex deploy keys), JWT and PEM shapes, JSON carrying private JWK members, high-entropy strings, and names ending _KEY, _SECRET, _TOKEN, _PASSWORD. An allowlist (OP_SERVICE_ACCOUNT_TOKEN plus explicit non-secret config) is the only escape; op:// references and Railway ${{ }} references pass. Findings name the variable and the reason, never the value. No export provided means a skip (advisory), so builds without one stay green. Standalone: pnpm verify:platform-env <file|->.

Threat modelCopy Anchor Link

RotationCopy Anchor Link

• Rotate a secret by editing its 1Password item, then restarting (redeploying) the app; no platform change is needed. Restart is required because values are read once at boot.
• Rotate the bootstrap token by creating a new service-account token, setting it on the platform, redeploying, then revoking the old one.
• Gateway tokens rotate themselves every 5 minutes. Rotating the signing key follows the existing Exo ID JWKS procedure (publish the new key first; Apps fetch JWKS by kid).
• The 2026-10-21 Railway advice to roll every variable is a separate, owner-approved action; this ADR does not rotate or revoke anything.

Blast radiusCopy Anchor Link

One service account per app environment, read-only, one vault. A leak of Exo Staging's token exposes only EXO staging's few secrets, never production, never another app. Today a Railway leak exposes every variable on every service in the project.

Cost and unit economicsCopy Anchor Link

• 1Password service accounts come with the 1Password plan; there is no per-service-account fee in the plan Exo already uses for people (owner brief, 2026-10-21; the roadmap's 2026-10-11 survey lists Business at $8.99 per user per month). Not re-verified today: 1Password's Business pricing page did not render for the check.
• Rate limits are the real constraint (1Password rate limits): Teams 1,000 reads per hour per token and 5,000 requests per day per account; Business 10,000 reads per hour per token and 50,000 per day per account. Exo Secure makes one batched read per boot, so even 100 deploys and restarts a day across every app is far inside Teams limits.
• Cost per unit: $0 marginal per app, per deploy and per request. Gateway tokens cost one Convex HTTP action per App every ~4 minutes while traffic flows (about 360 calls per App per day), well inside the Convex plan's included function calls. The platform-env check adds well under a second to the gate.
• Margin note: Exo Secure is not a cost driver, so it is not a pricing metric; include it in every plan, unmetered. If Exo later hosts vaults for customers, cost scales per human seat on 1Password, so price per seat, never per app or per secret.

Migration tableCopy Anchor Link

Every secret-shaped variable the exo repo reads today (grep of process.env and app.env across src/, convex/, packages/, scripts/, plus .env.example), with its destination. Owner is Samir (Interspace) for every row unless noted; the provider column is who issues the value.

ConsequencesCopy Anchor Link

• EXO's Railway service ends with one secret-shaped variable. Every other managed app adopts the same module (copy src/lib/exo-secure) and the same gate check.
• A boot now depends on 1Password being reachable; a failed deploy stays on the previous version.
• Secrets change only on restart. Hot reload of secrets is out of scope.
• Open items for the owner: create the vaults and service accounts; choose Teams or Business (rate limits); approve option A for Convex; decide how the build-time EXO_VERIFY_TOKEN is replaced.