Reference

CRM Sync — Key Management Lifecycle

Version: 1.5 Date: 2026-07-03 (v1.4: 2026-06-22; v1.2: 2026-06-15; v1.1: 2026-06-11; v1.0: 2026-05-26) Scope: Dev → Stage → Prod key management, consulting team workflow, stakeholder publish, per-market site tokens, Interactive Key Ceremony (agent-safe credential operations), .env/CLI-as-AI-exposure-surface threat model, organizational RACI + tech→business ownership, regulated-industry glossary, GraphQL/server-side migration significance, self-service tenant key management (app owners, §13), server-side session revocation Companion: KEY-CEREMONY-LOOP-REVIEW.md — recurring, non-custodial automation checklist that continuously attests this lifecycle (the QA/Compliance verify lane as a /loop).


0. Executive Summary (Challenge → Solution)

Each pillar of this lifecycle stated as one plain challenge/solution sentence, with the supporting detail bulleted beneath. Section references point to the full treatment.

Agent-safe key ceremony (§9). Challenge: an AI/automation agent is a superb operator but a liability as a secret-holder — once it can mint or read a key, that secret's reach includes its context window, the transcript, and every log it touches. Solution: run every privileged credential op as a two-role ceremony where a human executes and the agent only prepares, verifies, and records.

.env + CLI as an AI exposure surface (§9.6). Challenge: the everyday way to manage secrets — a .env file plus wrangler/curl at the terminal — is exactly what leaks to an agent sharing the machine. Solution: treat .env and the CLI as hostile-to-AI by default and route every secret around the agent.

Organizational roles & RACI (§9.7, §12). Challenge: a bare two-person ceremony doesn't say who does what once a real org is involved. Solution: map functional roles onto the ceremony with a RACI matrix that has exactly one Accountable per activity.

Formal definitions for regulated industry (§10). Challenge: auditors won't accept colloquial terms — controls must map to named standards. Solution: a glossary giving each term its normative definition and citation.

GraphQL / server-side migration significance (§11). Challenge: Shopify→GraphQL and Google→GA4-server-side concentrate credential power into fewer, high-privilege, server-resident secrets — just as AI agents do more of the integration work. Solution: make least-privilege scoping + rotation + the ceremony load-bearing controls at the worker boundary.

Ownership, tech → business (§12). Challenge: without a single owner, accountability diffuses and the control silently rots. Solution: name one Accountable per activity, escalating from execution on the technical floor to business risk ownership.

Self-service tenant key management (§13). Challenge: every key reset routed through the platform operator makes the operator a bottleneck — and a custodian — for credentials that rightfully belong to each app owner. Solution: the purchased product includes the key system: owners run their own lifecycle through an entitlement-gated wizard; the platform demonstrates it but never holds the key.


1. Environments

EnvironmentWorker NameDomainShopify StorePurpose
Devlocal (wrangler dev)localhost:8787—Local development, no real secrets
Stagecf-worker-crm-synccrm.story-story.aihx-stage.myshopify.comIntegration testing, client demos
Prodcf-worker-crm-sync-prod (or client's worker)crm.{client-domain}{client}.myshopify.comLive customer traffic

2. Key Types

KeyStorageScopeRotatable via APIEnv-specific
Root ADMIN_KEYCloudflare secretWorker-wideNo (CLI only)Yes — different per env
Rotatable admin keyKV admin_key:rotatableWorker-wideYes (POST /admin/rotate-key)Yes — lives in env's KV
Tenant token (crm_t_*)KV tenant_token:{token}Shop-scopedProvision/revoke via APIYes — lives in env's KV
JWT_SECRETCloudflare secretSession signingNo (CLI only)Yes — different per env
GOOGLE_CLIENT_SECRETCloudflare secretOAuthNo (CLI only)Yes — per Google project
SHOPIFY_ADMIN_TOKENKV (per-tenant config)Shop-scopedAuto-rotated (expiring tokens)Yes — per Shopify app
XANO_API_KEYCloudflare secret or KVDatabase accessNoYes — per Xano workspace
RESEND_API_KEYCloudflare secretTransactional emailNoYes — per Resend account
GA4_API_SECRETCloudflare secret or KVAnalyticsNoYes — per GA4 property
WEBFLOW_CMS_TOKENKV (auto-set by OAuth)CMS writesAuto-set by Webflow OAuthYes — per Webflow site
Market site tokenKV market_cms_token:{region}One market's Webflow siteRe-set via API (write-only)Yes — one per market clone

3. Dev Environment

Who: Engineering team (Persona A)

# Start local dev
cd workers/crm-sync
wrangler dev --config wrangler.toml

Key rules:

.dev.vars example:

ADMIN_KEY=dev-only-throwaway-key-1234567890
JWT_SECRET=dev-jwt-secret-not-for-production
XANO_API_KEY=xano-dev-key
GOOGLE_CLIENT_SECRET=google-dev-secret
RESEND_API_KEY=re_test_xxxxx

Extension dev:

cd extensions/crm-auth
npm run dev    # webflow extension serve 1338 + tsc --watch

4. Stage Environment

Who: Engineering + consulting team Worker: cf-worker-crm-sync on crm.story-story.ai Store: hx-stage.myshopify.com

4.1 Initial Stage Setup

# Set stage secrets (one-time)
cd workers/crm-sync
openssl rand -hex 24 | wrangler secret put ADMIN_KEY --config wrangler.toml
openssl rand -hex 24 | wrangler secret put JWT_SECRET --config wrangler.toml
wrangler secret put GOOGLE_CLIENT_SECRET --config wrangler.toml   # from Google Console
wrangler secret put XANO_API_KEY --config wrangler.toml           # from Xano stage workspace
wrangler secret put RESEND_API_KEY --config wrangler.toml         # from Resend (test mode)

4.2 Stage Key Inventory

KeyValue SourceWho Holds It
Root ADMIN_KEYGenerated at deployEngineering lead only
Rotatable keyCreated via API for consulting teamConsulting team lead
Tenant token (crm_t_*)Provisioned per demo clientConsulting team member
Google OAuthstory-story Google projectEngineering lead
Shopify AdminAuto-set by OAuth install on hx-stageWorker (KV)

4.3 Consulting Team Stage Access

# Engineering lead creates rotatable key for consulting team
curl -X POST https://crm.story-story.ai/admin/rotate-key \
  -H "Authorization: Bearer $ROOT_ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{"key":"consulting-stage-key-minimum-20chars"}'

# Consulting team uses this key for all stage admin operations
# They CANNOT access root key or CLI — only API-level operations

Consulting team can:

Consulting team cannot:

Verified in practice (2026-06-11): the full delegation lifecycle — root-authenticated create, masked status check (GET /admin/rotate-key returns a key_preview only, never the key), delegate authentication on admin endpoints, self-rotation, and root-only revocation (DELETE /admin/rotate-key) — is exercised and confirmed on stage. Rotating or revoking the delegate key never invalidates the root key or tenant tokens.

4.4 Market Site Tokens (multi-site localization)

Each localized market (e.g. a per-country Webflow site clone) authenticates with its own site-scoped token, because Webflow site tokens cannot cross sites:

# Store a market's site token (write-only field; created in that site's
# Webflow settings → Apps & Integrations → API access)
curl -X POST https://crm.story-story.ai/admin/markets \
  -H "Authorization: Bearer $ADMIN_OR_DELEGATE_KEY" \
  -H "Content-Type: application/json" \
  -d '{"region":"ca","cms_token":"<site token>"}'

Security properties:


5. Prod Environment

Who: Depends on stakeholder tier

5.1 Persona A (App Creator) — Prod Deploy

# Create production worker (separate from stage)
cd workers/crm-sync
cp wrangler.toml wrangler.prod.toml
# Edit wrangler.prod.toml: name = "cf-worker-crm-sync-prod", new KV namespace ID

# Set production secrets
openssl rand -hex 32 | wrangler secret put ADMIN_KEY --config wrangler.prod.toml
openssl rand -hex 32 | wrangler secret put JWT_SECRET --config wrangler.prod.toml
wrangler secret put GOOGLE_CLIENT_SECRET --config wrangler.prod.toml
wrangler secret put XANO_API_KEY --config wrangler.prod.toml
wrangler secret put RESEND_API_KEY --config wrangler.prod.toml

# Deploy
wrangler deploy --config wrangler.prod.toml

# Set up custom domain
# (via Cloudflare Workers Custom Domains API or dashboard)

5.2 Persona B (Shared) — Prod Onboarding

# A provisions tenant token for B's store
curl -X POST https://crm.{prod-domain}/admin/provision-token \
  -H "Authorization: Bearer $PROD_ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "shop": "client-store.myshopify.com",
    "label": "client-name",
    "scopes": ["config:read", "config:write"]
  }'
# → Returns: crm_t_xxxx

# B receives ONLY:
#   1. Tenant token (crm_t_xxxx)
#   2. Worker URL (https://crm.{prod-domain})
#   3. Extension auto-fills the URL
# B never sees: root key, Cloudflare secrets, KV internals

5.3 Persona C (Private Worker) — Prod Handoff

# C deploys their own worker from the source code license
cd workers/crm-sync
wrangler deploy --config wrangler.toml   # their own Cloudflare account

# C sets their own root key
openssl rand -hex 32 | wrangler secret put ADMIN_KEY --config wrangler.toml

# C creates a rotatable key for day-to-day operations
curl -X POST https://their-worker.their-domain.com/admin/rotate-key \
  -H "Authorization: Bearer $THEIR_ROOT_KEY" \
  -H "Content-Type: application/json" -d '{}'
# → Returns auto-generated rotatable key

# C provisions tenant tokens for each of their Shopify stores
curl -X POST https://their-worker.their-domain.com/admin/provision-token \
  -H "Authorization: Bearer $ROTATABLE_KEY" \
  -H "Content-Type: application/json" \
  -d '{"shop":"store-1.myshopify.com","label":"store-1"}'

curl -X POST https://their-worker.their-domain.com/admin/provision-token \
  -H "Authorization: Bearer $ROTATABLE_KEY" \
  -H "Content-Type: application/json" \
  -d '{"shop":"store-2.myshopify.com","label":"store-2"}'

6. Consulting Team → Live Deploy Workflow

6.1 Pre-Deploy Checklist

#TaskEnvironmentWho
1Verify all integrations on stageStageConsulting team
2Document client's required OAuth scopesStageConsulting team
3Request prod deploy from engineering—Consulting team → A
4Create prod worker or provision on sharedProdA (engineering)
5Set prod secretsProdA (engineering)
6Create consulting rotatable key for prodProdA → consulting team
7Provision client tenant tokenProdConsulting team
8Client installs Shopify app (OAuth flow)ProdClient (B)
9Client connects Google OAuth in extensionProdClient (B)
10Verify end-to-end auth flowProdConsulting team
11Rotate consulting key (remove team access)ProdA or consulting lead

6.2 Key Handoff Matrix

                    ┌─────────────────────────────────────────────┐
                    │           ENGINEERING (A)                    │
                    │  Root ADMIN_KEY (prod)                       │
                    │  All Cloudflare secrets                      │
                    │  Google Console, Xano, Resend accounts       │
                    └────────────┬────────────────────────────────┘
                                 │
                    Creates rotatable key
                                 │
                    ┌────────────▼────────────────────────────────┐
                    │        CONSULTING TEAM                       │
                    │  Rotatable admin key (prod)                  │
                    │  Can: provision tokens, manage config,       │
                    │       access setup/settings, rotate own key  │
                    │  Cannot: deploy, change secrets, root access │
                    └────────────┬────────────────────────────────┘
                                 │
                    Provisions tenant token
                                 │
              ┌──────────────────┼──────────────────┐
              │                                      │
   ┌──────────▼──────────┐              ┌────────────▼───────────┐
   │    CLIENT (B)        │              │    CLIENT (C)           │
   │  Tenant token only   │              │  Full source + own key  │
   │  crm_t_xxxx          │              │  Root + rotatable       │
   │  Extension auto-fill │              │  Own infra, own deploy  │
   │  No admin access     │              │  Multi-tenant capable   │
   └──────────────────────┘              └────────────────────────┘

6.3 Post-Launch Key Rotation

After consulting team completes handoff:

# Option 1: Rotate consulting key (new key, team retains support access)
curl -X POST https://crm.{prod-domain}/admin/rotate-key \
  -H "Authorization: Bearer $CONSULTING_KEY" \
  -H "Content-Type: application/json" -d '{}'
# Old consulting key instantly invalid, new key for ongoing support

# Option 2: Revoke consulting key entirely (root-only)
curl -X DELETE https://crm.{prod-domain}/admin/rotate-key \
  -H "Authorization: Bearer $ROOT_ADMIN_KEY"
# Only root key valid — consulting team fully cut off

# Option 3: Issue new consulting key with different holder
curl -X POST https://crm.{prod-domain}/admin/rotate-key \
  -H "Authorization: Bearer $ROOT_ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{"key":"new-support-team-key-at-least-20"}'

7. Stakeholder Publish — Key Inventory

7.1 What Each Stakeholder Receives at Publish

CredentialA (Creator)B (Shared)C (Private)
Root ADMIN_KEY✅ Generated + held❌ Never✅ They generate
Rotatable key✅ Optional❌ Never✅ Self-service
Tenant tokenN/A (is the admin)✅ crm_t_*✅ For sub-tenants
Worker URLOwn domainProvided (auto-fill)Own domain
Extension URL/extension/ on own workerSame (A's worker)/extension/ on own worker
Google Client IDOwn projectShared (A's)Own project
Google Client SecretOwn project❌ NeverOwn project
Shopify Admin TokenAuto via OAuthAuto via OAuthAuto via OAuth
Xano API KeyOwn workspace❌ NeverOwn workspace
Cloudflare dashboard✅ Full access❌ Never✅ Own account
Source code✅ Full repo❌ Never✅ Licensed copy

7.2 Publish Ceremony (Per Stakeholder)

Stakeholder B publish:

  1. A provisions crm_t_* token scoped to B's shop
  2. A sends B: tenant token + worker URL
  3. B installs Webflow extension → auto-fills worker URL
  4. B enters tenant token in Config tab
  5. B installs Shopify app (OAuth flow auto-provisions admin token)
  6. B connects Google login via extension Auth tab
  7. Done — B operates entirely through extension UI

Stakeholder C publish:

  1. A delivers licensed source code repo
  2. C deploys to their Cloudflare account (wrangler deploy)
  3. C sets root ADMIN_KEY via CLI
  4. C creates rotatable key via API (for team operations)
  5. C sets up their own: Google project, Xano workspace, Resend account
  6. C registers their Shopify app in Partners
  7. C updates Webflow extension URL to their worker domain
  8. C provisions tenant tokens per Shopify store
  9. C sets plan: "private" in config
  10. Done — C is fully independent, A has no access

8. Security Policies

8.1 Key Rotation Schedule

KeyRotation FrequencyWho RotatesMethod
Root ADMIN_KEYAnnually or on personnel changeA (engineering)wrangler secret put
Rotatable keyQuarterly or on team changeKey holderPOST /admin/rotate-key
Tenant tokensOn client offboarding or breachA or consulting teamPOST /admin/revoke-token
JWT_SECRETAnnually (invalidates all sessions)A (engineering)wrangler secret put
Google Client SecretOn suspected compromiseA (engineering)Google Console + wrangler secret put
Shopify Admin TokenAuto-rotated (expiring tokens)Worker (cron)Automatic

8.2 Incident Response — Key Compromise

ScenarioImmediate ActionRecovery
Root key leakedwrangler secret put ADMIN_KEY with new valueRotate all dependent keys, audit KV access logs
Rotatable key leakedPOST /admin/rotate-key (self-rotation)Audit what was accessed, notify affected clients
Tenant token leakedPOST /admin/revoke-token + provision newNotify affected client, review shop config for tampering
JWT_SECRET leakedwrangler secret put JWT_SECRETAll user sessions invalidated — users must re-login
Consulting key leakedDELETE /admin/rotate-key (root revoke)Audit operations during exposure window

8.3 Environment Isolation Rules


9. Interactive Key Ceremony (Agent-Safe Credential Operations)

Primary feature. Every privileged credential operation in this stack — mint, rotate, revoke, set — runs as a two-role ceremony: a named Security Human executes the privileged command; an Agent (AI/automation, e.g. Claude Code) prepares, verifies, and records but never mints, holds, or transports a secret. This is the normative procedure behind FUNCTIONAL-SPEC.md §13 FR-HANDOFF-02.

9.0 Heritage — an established discipline, applied to a new threat

"Key ceremony" is not a term invented for this project. It is a long-standing practice in cryptography and high-assurance operations:

The classic ceremony defends against a malicious or careless human insider. Our contribution is to extend the same primitives — split roles, dual control, verify-don't-view, fingerprint-only audit — to a new untrusted party: the AI/automation agent. The "Interactive Key Ceremony" treats a capable coding/ops agent the way a CA treats any single custodian: useful for everything around the secret, but never the holder of it. Framing it this way is deliberate — this is a known discipline applied to the agentic-operations threat surface, not a bespoke coinage.

9.1 Why this exists

A capable coding/ops agent is a force multiplier for everything except secret custody. The moment an agent can mint or read a production credential, that secret's blast radius includes the model's context window, the transcript, and any log it touches. The ceremony removes the agent from the secret path by construction — not by policy reminder — so the agent stays maximally useful (it writes the runbook, runs every verification probe, files the audit record) while the secret never enters a surface it shouldn't.

Three properties make it safe by construction, not by good behaviour:

  1. The agent never executes the privileged write. A permission classifier blocks credential-minting commands; only the Security Human's interactive invocation runs it. A blocked mint is handed to the operator — never worked around.
  2. The secret never enters the transcript. The new key's value is written straight to disk (~/.crm-*-key, chmod 600) or piped directly into wrangler secret put. Read-back surfaces return a masked preview (564bbe…9298) or a "(set)" flag — never the value.
  3. Additive-only, verified-first. The agent reads the endpoint's source first to confirm the operation is additive (the rotatable key lives at KV admin_key:rotatable, alongside — never replacing — the root ADMIN_KEY), so a failed or interrupted ceremony can never orphan the root credential.

9.2 The ceremony flow

  ┌──────────────┐        ┌─────────────────────┐        ┌──────────────────┐
  │ SECURITY     │        │  AGENT               │        │ PERMISSION       │
  │ HUMAN        │        │  (AI / automation)   │        │ CLASSIFIER       │
  └──────┬───────┘        └──────────┬───────────┘        └────────┬─────────┘
         │                           │                             │
         │   1. "rotate the key"     │                             │
         │ ─────────────────────────▶│                             │
         │                           │  2. READ source first       │
         │                           │     (endpoint additive?     │
         │                           │      root untouched?)       │
         │                           │                             │
         │                           │  3. attempt the mint  ──────▶│
         │                           │                             │ 4. BLOCK
         │                           │◀──────  denied  ────────────│   (credential
         │                           │                             │    minting)
         │   5. agent hands command  │                             │
         │◀──── back to operator ────│                             │
         │                           │                             │
   6. RUN interactively              │                             │
   curl -X POST …/admin/rotate-key   │                             │
   -H "Authorization: Bearer         │                             │
      $(cat ~/.crm-admin-key)"       │                             │
         │                           │                             │
   7. response field ──▶ written     │                             │
      straight to ~/.crm-*-key       │                             │
      (chmod 600, never printed)     │                             │
         │                           │                             │
         │   8. "done"               │                             │
         │ ─────────────────────────▶│                             │
         │                           │  9. VERIFY (no secret seen): │
         │                           │   • GET /admin/rotate-key →  │
         │                           │     masked preview only      │
         │                           │   • probe real endpoint      │
         │                           │     (/admin/markets → 200)   │
         │                           │   • old key → 401 (dead)     │
         │                           │   • check rotations: N meta  │
         │                           │                             │
         │                           │ 10. RECORD audit entry       │
         │                           │     (fingerprint only,       │
         │                           │      no key value)           │
         ▼                           ▼                             ▼

9.3 The trust boundary (who-can-touch-what)

Every arrow that creates a secret is executed by the Human; every arrow that checks one is executed by the Agent. The root key never leaves the Cloudflare secret store — rotation happens one rung down (admin_key:rotatable), which is why a botched ceremony can't lock anyone out.

   ROOT  ── Cloudflare secret ─────────────  CLI only, Security Human, never moves
    │      (ADMIN_KEY)
    │  POST /admin/rotate-key  (Human runs interactively; Agent blocked)
    ▼
 ROTATABLE ── KV: admin_key:rotatable ─────  delegate / agency, self-rotates,
    │                                         root-revocable (DELETE)
    │  POST /admin/provision-token
    ▼
 TENANT (crm_t_*) ── KV per-shop ──────────  client B, shop-scoped, no admin access

   AGENT scope ▒▒▒▒  verify + record only — never mints, never holds a value

9.4 Audit record format

Every ceremony files one append-only record. It carries fingerprints, never secrets (sha256(value)[:8]), so the trail is itself safe to publish:

FieldExampleNotes
date2026-06-11T16:49:00ZUTC, ceremony completion
credentialadmin_key:rotatablelogical name, not value
actionrotatemint \rotate \revoke \set
executed_bysecurity-human:yslwho ran the interactive command
verified_byagent:claude-codewho probed old-dead / new-alive
old_fp → new_fp1a2b3c4d → 564bbe92sha256[:8]; proves change without exposure
reasonquarterly-cadencehandoff \90d \personnel \incident \scope
overlap_window0s (instant cutover)how long both keys were valid
consumers_updatedworker secret; Actions secretevery surface that had to change

9.5 Operator rules (reaffirmed)

9.6 Challenge & solution — .env files and the CLI as an AI exposure surface

The conventional way to manage secrets across Cloudflare, Shopify, and Xano — a .env file on disk plus a wrangler secret put typed at the terminal — is exactly the surface that leaks to an AI agent. The ceremony exists because these two everyday conveniences become liabilities the moment a capable agent shares the workstation.

The challenge (three concrete exposure vectors):

The solution (route every secret around the agent):

VectorMitigation in this stack
.env plaintextPrivileged secrets are never kept in a checked-out .env. Worker secrets live as Cloudflare secrets (encrypted, write-only — settable but never readable back); tenant keys live in KV; local custody is chmod 600 dotfiles (~/.crm-admin-key) read only by the human. Dev uses .dev.vars with throwaway keys (§3).
Inline CLI / historyThe human runs the privileged command interactively; secrets are injected by reference — $(cat ~/.crm-admin-key) — never typed inline, never printed, never committed. A permission classifier blocks the agent from executing credential-minting commands and hands them to the operator.
Echo-back on verifyRead-back surfaces return a masked preview (564bbe…9298) or a "(set)" flag — never the value. The agent verifies by side-effect: a 200 on a real endpoint, an old key now returning 401, a rotation-count delta, a fingerprint diff (§9.4).
Config-layer overridesRotation happens in KV/tenant config (which wins over env), so a key change needs no .env edit and no redeploy — the value never passes through a build artifact or env file.

Net posture: the .env file and the CLI are treated as hostile-to-AI by default. The human is the only party that ever handles a secret's value; the agent is the only party that keeps the (value-free, fingerprint-only) audit trail. Usefulness of the agent is preserved; custody of the secret is structurally withheld from it.

9.7 Ceremony roles in an organization (RACI)

The two-role model (§9.2) is a minimum: one Security Human + one Agent. In a staffed organization those duties distribute across named functional roles. The table below maps each to the ceremony — and which existing actor in §9.2 it embodies.

Two persona dimensions apply:

Organizational personaOrg mandateRole in the ceremonyArtifact ownedMaps to §9.2 actor
Compliance OfficerRegulatory adherence (PCI-DSS Req. 3 key mgmt, SOC 2 CC6.1)Attests rotation cadence is met; reviews the audit log (§9.4); signs incident-closure after a compromise (§8.2)Rotation schedule (§8.1) + audit-record sign-offOversight — neither executes nor holds; consumes the fingerprint trail
DPO (Data Protection Officer)GDPR Art. 37–39; protects the personal-data / PII planeApproves any ceremony whose blast radius includes personal data — JWT_SECRET (sessions) and XANO_API_KEY (PII data store) — before it runsData-protection sign-off / DPIA noteApproval gate upstream of the Security Human
PMO (Project Mgmt Office)Coordinates handoffs and cadence as scheduled eventsInitiates the ceremony ("rotate the key"), owns this RACI, tracks the 90-day / handoff / personnel triggersCeremony calendar + handoff runbook (AGENCY-HANDOFF.md)Coordinator — the request that opens step 1 of §9.2
QA / Release ManagerRelease quality gateVerifies the change on Stage before Prod; issues go/no-go that a rotation didn't break a release; co-signs the Agent's verification (§9.2 step 9)Stage verification result + go/no-go recordVerifier — pairs with the Agent's probes
Deployment OfficerExecutes privileged writes against the live environmentRuns the interactive command — wrangler secret put, curl …/admin/rotate-key — with the secret injected by reference; holds chmod 600 dotfile custodyThe executed command + new secret on diskSecurity Human (executor) — the only role that touches a value

**The Agent (AI/automation) maps to none of the human roles.** It prepares the runbook, runs the verification probes on the QA/Compliance officers' behalf, and files the fingerprint-only audit record — but never executes the privileged write and never holds a value (§9.1).

9.7.1 Segregation of duties (the point of dual control)

The compliance value only holds if the roles do not collapse into one person:


10. Formal Definitions (Regulated-Industry Glossary)

These are the normative, citable definitions a regulated deployment (PCI-DSS, SOC 2, GDPR, ISO 27001, FIPS-validated environments) will expect when auditing this lifecycle. Where this project's terms are colloquial (e.g. "blast radius"), the formal equivalent is given so the control maps cleanly to an assessor's framework.

Term (as used here)Formal definitionAuthoritative source
Key ceremonyA formally scripted, witnessed, and audited procedure for executing a sensitive key-management operation (generation, rotation, revocation, distribution) under predefined controls and role separation.NIST SP 800-57 Pt 2 (key-mgmt org); ICANN DNSSEC DPS; PCI PIN Security
Dual controlA process requiring two or more separate, authorized individuals to act together to perform a single operation, such that no one individual can perform it alone.PCI-DSS Glossary; PCI PIN Security Req.
Split knowledgeA condition in which two or more parties separately hold key components that, individually, convey no knowledge of the resultant cryptographic key.PCI-DSS Glossary; NIST SP 800-57 Pt 1
Separation / Segregation of Duties (SoD)A control dividing a sensitive task across multiple actors so no single actor can both execute and conceal an action; reduces fraud and undetected error.NIST SP 800-53 AC-5; ISO/IEC 27001:2022 A.5.3; SOC 2 CC
Least privilegeGranting each actor (human, service, or token) only the minimum access required to perform its function.NIST SP 800-53 AC-6
Key custodianAn individual formally entrusted with, and accountable for protecting, a cryptographic key or key component (typically with a signed custodian acknowledgment).PCI-DSS Req. 3 (key management)
Crypto Officer (CO)A defined operator role authorized to perform initialization and key-management functions on a cryptographic module. (Distinct from the "User" role.)FIPS 140-3 / ISO/IEC 19790
CryptoperiodThe bounded time span during which a specific key is authorized for use; expiry mandates rotation.NIST SP 800-57 Pt 1
Key rotationRetiring an active key at the end of its cryptoperiod (or on compromise) and replacing it with a new key, without loss of service.NIST SP 800-57 Pt 1
Scope of compromise (our "blast radius")The complete set of resources, data, and operations reachable by a single credential if disclosed — the unit of risk that least-privilege and rotation bound.NIST SP 800-53 (impact/containment)
Key fingerprintA truncated cryptographic hash of a key used to identify it (in logs/audits) without revealing the key material. Here: sha256(value)[:8] (§9.4).NIST SP 800-57 (key identifiers)
Non-repudiation / audit trailA tamper-evident, attributable record sufficient to prove who performed an action and that it occurred.SOC 2 CC7; GDPR Art. 30 (records of processing)
RACIA responsibility-assignment matrix classifying each actor against an activity as Responsible (does the work), Accountable (the single owner who answers for the outcome — exactly one per activity), Consulted (two-way input before action), or Informed (one-way notification after). Variants: RASCI (+ Support), RACI-VS (+ Verifies / Signs-off), CAIRO/RACIO (+ Omitted/out-of-loop).PMI PMBOK Guide; ISO 21500
Data Protection Officer (DPO)A mandated independent role responsible for monitoring an organization's personal-data protection compliance. Must not also determine the purposes/means of processing (conflict-of-interest bar) — hence advisory, never Accountable.GDPR Art. 37–39
Security of processingThe obligation to implement appropriate technical measures — incl. encryption and the ability to ensure ongoing confidentiality — proportionate to risk.GDPR Art. 32
Write-only secretA credential surface that accepts a value on set but never returns it on read (read-back yields a mask or (set) flag); prevents echo-back disclosure.Project control (§9.6); aligns NIST SP 800-57 protection-of-secrets

Mapping note for assessors. §9 (Interactive Key Ceremony) is the implementation of dual control + SoD + least privilege, extended so that an AI/automation agent is treated as a non-custodial actor — it may verify and record but never satisfies the "authorized individual" requirement for the privileged write. §9.4 (audit record) satisfies non-repudiation; §8.1 (rotation schedule) satisfies cryptoperiod bounds.


11. Significance in Recent Shopify & Google GraphQL / Server-Side Migrations

The two platform migrations this stack rides on — Shopify REST → GraphQL Admin and Google UA → GA4 (server-side measurement) — are not just API churn. Both move credential-bearing work from many low-privilege, client-exposed identifiers to fewer, high-privilege, server-side secrets concentrated in the edge worker. That concentration is exactly what makes the key lifecycle in this document load-bearing rather than optional. See SHOPIFY-2026-RISK-BRIEF.md and FEATURE-SPEC-UA-MIGRATION.md.

11.1 Shopify: REST → GraphQL Admin API

11.2 Google: Universal Analytics → GA4 (server-side measurement)

11.3 The common thread (why this document matters now)

Before (legacy)After (GraphQL / server-side)Consequence for key mgmt
Many REST routes, scoped per callOne GraphQL endpoint, one tokenLarger scope of compromise per credential → least-privilege + rotation are primary controls
Public client-side IDs (gtag, Storefront)Server-side secrets in the workerSecrets concentrate at the edge → §9.6 .env/CLI threat model is live, not theoretical
Static, long-lived tokensExpiring / rotating tokensCryptoperiod (§10) is enforced by the platform → §8.1 cadence aligns with vendor direction
Humans wrote integrationsAI agents generate GraphQL/codegen + run server-side opsThe automation building the integration must be non-custodial → §9.1 / §9.7 keep it in the verify-and-record lane

Bottom line: GraphQL's single-endpoint model and Google's server-side measurement both concentrate credential power into fewer, server-resident secrets — at the same moment AI agents are doing more of the integration work. That intersection is the reason the Interactive Key Ceremony (formal dual control + SoD, extended to treat the agent as non-custodial) is the right — and arguably required — control for a regulated deployment of this stack.


12. Ownership — Who Owns This, From Tech to Business

The RACI model (defined in §10) requires that every activity have exactly one Accountable owner. This section names that owner end-to-end and shows how ownership escalates from the technical floor to the business.

Legend: R = Responsible (does it) · A = Accountable (owns the outcome, one only) · C = Consulted (input before) · I = Informed (told after).

12.1 The single point of accountability

12.2 RACI across the lifecycle (tech → business)

ActivityDeployment OfficerEngineering LeadQA / Release MgrPMOCompliance OfficerDPOCISO (business)
Build / maintain the mechanism (worker, rotate endpoints)CA/RCIIII
Schedule a ceremony (cadence / handoff / incident trigger)ICCA/RCCI
Approve a ceremony touching the PII plane (JWT_SECRET, XANO_API_KEY)ICIICRA
Execute the privileged write (mint / rotate / revoke / set)A/RCIIIII
Verify old-dead / new-alive (non-secret probes)CCA/R (with Agent support)IIII
Stage-before-prod go / no-goICA/RCIII
Review audit record + attest cadence metIICIA/RCI
Incident response on key compromise (§8.2)RRCCRCA
Own residual risk / sign the control as effectiveICIICCA/R

Reading the chain: technical Responsibility concentrates on the Deployment Officer (execution) and Engineering Lead (mechanism); operational coordination on the PMO; assurance on QA and Compliance; lawful-basis advice on the DPO; and ultimate Accountability for risk on the CISO / business. Every row has exactly one A — the model's core invariant.

12.3 Smaller-org collapse (and the lines that must not collapse)

In a lean team one person may hold several letters, except the two invariants from §9.7.1: the executor (Deployment Officer) and the attester (Compliance Officer) must stay distinct, and Accountability (A) must sit with a business risk owner, not the person doing the work (R). If a single founder wears every hat, the Agent's non-custodial verify-and-record role becomes the de-facto second control — which is why the ceremony is designed to hold even at headcount = 1.


13. Self-Service Tenant Key Management (App Owners)

Added v1.5 (2026-07-03). The buyer-facing tier of the key hierarchy: every app purchase includes a dedicated per-store admin credential that the owner — not the platform — mints, rotates, revokes, and audits.

13.1 Position in the key hierarchy

TierCredentialHeld byReset blast radius
Platform rootworker root admin keyPlatform operator onlyEverything (full dependent rotation)
Delegaterotatable admin keyPlatform / consulting opsThe delegate only
App ownerper-store tenant tokenThe purchasing owner onlyThat one store only

The platform root and delegate tiers are covered by §§3–9. This section covers the third tier, which is the one sold with the product: the owner's dedicated key.

13.2 The owner lifecycle (wizard)

Owners manage their key through a guided wizard — sign in, confirm the store, generate. The design constraints mirror the ceremony's invariants, translated to self-service:

  1. Entitlement-gated ownership. A signed-in identity may manage a store's keys only if its entitlement record — written by purchase, not self-serviceable — lists that store. A login alone never escalates to key access.
  2. Shown once. The key value is returned exactly once at mint. Every subsequent view is a truncated fingerprint (sha256[:8]) and a masked preview. Losing the key means rotating it — by design, there is nothing to "look up."
  3. Mint-before-revoke rotation. The replacement key is written before predecessors are deleted, so no failure mode leaves the store keyless (the self-service analogue of §9's additive-only rule).
  4. Incident revoke. Any single key can be killed by fingerprint without a replacement — the owner's kill switch for a suspected leak.
  5. Append-only audit. Every rotate/revoke appends a fingerprint-only record (date, action, actor, old → new fingerprints) to a per-store ledger the owner can read in the wizard — the same record shape a ceremony files (§9.5), produced automatically.

13.3 Session integrity underneath the wizard

The wizard is only as trustworthy as the login in front of it:

13.4 Why this is the product, not a feature

A SaaS contract asks the customer to trust the vendor's custody of credentials and a document's promise about it. This model inverts custody: the platform operates the key system — mint, rotation, revocation, audit — while the customer holds the key value. The operator cannot leak what it never held; the owner cannot be locked out of what only they possess. Peer teams extend the same property to their agents: agents act under scoped, revocable mandates and never hold the owner's key (§9.1's non-custodial rule, applied to the customer's own automation).