Harness and orchestration

Purpose

The Harness is the orchestrator: it turns cadences, integration events, IC messages, escalations, and handoffs into runs; executes those runs inside budgets and policy gates; records every step as an event; and guarantees the things the product promises regardless of what the model does. This document specifies the run model, the lifecycle state machine, idempotency, retries, dedup, ordering, event sourcing and replay, budgets, gates, concurrency and backpressure, the listening period and graduation, and gives the scheduler loop and run executor as pseudocode that is meant to be transcribed into code, not interpreted.

Scope / Non-scope

In scope

• RunRequest, Run, triggers, priority classes, subject keys.
• Lifecycle state machine and transition rules.
• Idempotency keys, dedup and coalescing, retries and backoff, ordering guarantees.
• Event envelope, step events, replay modes.
• Budgets (tokens, time, tool calls, cost, outreach) per run, per tenant, per person.
• The three policy gates (admission, action, send) and the effective-mode calculation.
• Per-tenant concurrency, backpressure levels, missed-run detection.
• Listening period, graduation state per function, regression.
• Scheduler loop and run executor pseudocode.

Non-scope

• What a Chonk does inside a step (planner/analyst/chaser/writer) → Manager Agents.
• The policy rule language → Policy Engine.
• Decision routing and approvals UX → Decision Pipeline, Escalation and Human-in-the-Loop.
• Metrics and dashboards for the Harness → Observability and Runbooks.
• Event payload schemas in full → Data Model and API Reference.

1. Run model

A run is the unit of work, isolation, budgeting, and audit. Every run has exactly one tenant, one Chonk, one trigger, one idempotency key, one subject key, one priority, one budget, and one pinned Org Model version.

type TriggerKind =
  | 'CADENCE_TICK' | 'INTEGRATION_EVENT' | 'IC_MESSAGE' | 'ESCALATION'
  | 'HANDOFF' | 'HUMAN_DECISION' | 'FOLLOWUP' | 'REPLAY' | 'MANUAL';

type Trigger =
  | { kind: 'CADENCE_TICK'; cadenceId: CadenceId; fireAt: ISODateTime; participantId?: PersonId }
  | { kind: 'INTEGRATION_EVENT'; provider: ConnectorProvider; deliveryId: string; eventType: string;
      subjectRefs: NodeRef[]; signal?: Signal }            // signal: detector-derived (stall, variance, uncovered shift)
  | { kind: 'IC_MESSAGE'; surface: Surface; threadId: string; messageId: string; personId: PersonId }
  | { kind: 'ESCALATION'; escalationId: EscalationId; stage: number }
  | { kind: 'HANDOFF'; handoffId: HandoffId; fromChonk: ChonkId; toChonk: ChonkId }
  | { kind: 'HUMAN_DECISION'; decisionId: DecisionId; status: 'confirmed' | 'rejected' | 'expired'; actorId?: PersonId; resumeRunId?: RunId }
  | { kind: 'FOLLOWUP'; parentRunId: RunId; stage: number; dueAt: ISODateTime }
  | { kind: 'REPLAY'; sourceRunId: RunId; mode: ReplayMode; nonce: string }
  | { kind: 'MANUAL'; actorId: PersonId; objective: string; clientNonce: string };

type Priority = 'P0' | 'P1' | 'P2' | 'P3' | 'P4' | 'P5';
// P0 safety / IC→human escalation / cadence.missed
// P1 IC messages, decision resumes, windowed atoms cadences (schedule publish, checklists)
// P2 windowed cadences (briefs, huddles, owner update)
// P3 signal-driven chases and follow-ups
// P4 reviews, planning, reporting
// P5 content renders, celebrations, narrative refresh

type RunKind =
  | 'morning_brief' | 'ic_reply' | 'checkin' | 'one_on_one' | 'chase' | 'followup'
  | 'weekly_review' | 'owner_update' | 'quarterly_planning' | 'replan' | 'approval'
  | 'schedule_publish' | 'shift_swap' | 'checklist' | 'escalation' | 'handoff'
  | 'onboarding_draft' | 'listening_intro' | 'content_render' | 'integrity_check' | 'custom';

interface RunRequest {
  runId: RunId;
  tenantId: TenantId;
  chonkId: ChonkId;
  runKind: RunKind;
  trigger: Trigger;
  idempotencyKey: string;         // §3
  subjectKey: string;             // §6: 'person:prs_…' | 'project:prj_…' | 'org:org_…' | 'tenant'
  priority: Priority;
  deadline?: ISODateTime;         // windowed cadences: window end
  expirable: boolean;             // true → drop past deadline (and fall back); false → run late
  coalescable: boolean;           // §4
  budget: RunBudget;              // §7
  requestedAt: ISODateTime;
  attempt: number;                // 0 on first request
  parentRunId?: RunId;
}

interface Run extends RunRequest {
  state: RunState;
  orgModelVersion: number;        // pinned at admission
  functionModes: Record<ManagementFunction, ExecutionMode>;   // computed at admission
  leaseId?: string;
  startedAt?: ISODateTime;
  endedAt?: ISODateTime;
  usage: Usage;                   // running totals against budget
  pendingDecisions: DecisionId[];
  failure?: { class: FailureClass; message: string; retryable: boolean };
  replay?: { sourceRunId: RunId; mode: ReplayMode };
}

Which Chonk gets the run is deterministic: cadences carry ownerChonkId; IC messages go to the Chonk that manages the person's primary org (managed_by); integration events and signals are routed by a static table keyed on eventType and the subject's org (a GitHub stall → build; a spend request → fin; an uncovered shift → ops); escalations go to Chonksworth for triage unless routed directly to a human; handoffs name their target. Nothing in this routing consults a model.

2. Run lifecycle

stateDiagram-v2
  [*] --> REQUESTED: enqueue()
  REQUESTED --> DEDUPLICATED: idempotency key exists
  REQUESTED --> COALESCED: merged into pending run
  REQUESTED --> QUEUED
  QUEUED --> DEFERRED: backpressure level excludes priority
  DEFERRED --> QUEUED: resume timer / level drops
  QUEUED --> EXPIRED: past deadline and expirable
  QUEUED --> ADMITTED: lease acquired, version pinned, admission gate passed
  QUEUED --> FAILED: admission gate BLOCK
  ADMITTED --> RUNNING: executor picked up
  RUNNING --> WAITING_HUMAN: PROPOSE_CONFIRM or HUMAN_ONLY pending
  RUNNING --> WAITING_EXTERNAL: async connector write / content render
  WAITING_HUMAN --> RUNNING: HUMAN_DECISION trigger (resume)
  WAITING_EXTERNAL --> RUNNING: callback
  WAITING_HUMAN --> SUCCEEDED: all decisions terminal
  RUNNING --> SUCCEEDED
  RUNNING --> BUDGET_EXCEEDED
  RUNNING --> FAILED_RETRYABLE
  RUNNING --> FAILED
  FAILED_RETRYABLE --> QUEUED: attempt+1 after backoff
  FAILED_RETRYABLE --> DEAD_LETTER: attempts exhausted
  BUDGET_EXCEEDED --> SUCCEEDED: fallback delivered
  BUDGET_EXCEEDED --> DEAD_LETTER: no fallback
  QUEUED --> CANCELLED: tenant paused / admin cancel
  WAITING_HUMAN --> CANCELLED: admin cancel (decisions stay open)
  DEDUPLICATED --> [*]
  COALESCED --> [*]
  EXPIRED --> [*]
  SUCCEEDED --> [*]
  FAILED --> [*]
  DEAD_LETTER --> [*]
  CANCELLED --> [*]

Transition rules that are easy to get wrong:

From → To	Rule
RUNNING → WAITING_HUMAN	The subject lease is released on entering WAITING so other runs on the subject proceed. Resumption acquires a new lease.
WAITING_HUMAN → RUNNING	Resumption re-reads the Org Model at the current version (not the pinned one) before executing a confirmed action, and re-classifies the action. A verdict that changed (e.g., the owner narrowed scope while the decision was pending) yields decision.stale, never a silent execution.
QUEUED → EXPIRED	Only expirable runs expire. Briefs expire at window end and trigger the fallback brief; chases never expire (they run late and say so).
FAILED_RETRYABLE → QUEUED	Same runId, same idempotency key, attempt+1. The event log shows one run with several attempts.
* → CANCELLED	Cancelling a run never cancels its decisions; open decisions remain in the inbox and resume into a fresh run (EC-ARCH-003 in System Architecture).
BUDGET_EXCEEDED → SUCCEEDED	Only when a deterministic fallback exists for the run kind (brief, huddle notes, checklist prompt, owner update text-only).

3. Idempotency keys

idempotencyKey = sha256(tenantId + '|' + material) with material by trigger:

Trigger	Material	Notes
CADENCE_TICK	cad:{cadenceId}:{participantId or '-'}:{fireAt truncated to minute}	A cadence firing twice in the same minute (leader failover) is one run.
INTEGRATION_EVENT	int:{provider}:{deliveryId} and, for detector signals, int:{provider}:{deliveryId}:{ruleId}	Provider delivery IDs are unique per delivery; providers that retry reuse the ID. Polled events synthesize deliveryId = hash(objectId, updatedAt).
IC_MESSAGE	msg:{surface}:{threadId}:{messageId}	Slack ts, Teams message id, Twilio MessageSid.
ESCALATION	esc:{escalationId}:{stage}	Each stage of an escalation ladder is a distinct run.
HANDOFF	hnd:{handoffId}
HUMAN_DECISION	dec:{decisionId}:{status}	Double-tap on Approve is one resume.
FOLLOWUP	fup:{parentRunId}:{stage}
REPLAY	rpl:{sourceRunId}:{nonce}	Never deduplicated; the nonce is new per replay request.
MANUAL	man:{actorId}:{clientNonce}	Client supplies the nonce (UI generates a UUID per click).

Keys live in run_idempotency (tenant_id, key, run_id, created_at) with a unique index and a 7-day TTL. The insert is in the same transaction as the run.requested event; a conflict emits run.deduplicated with the original runId so the caller can attach to it.

4. Dedup and coalescing

Dedup is exact-key. Coalescing is semantic and only for coalescable run kinds:

Run kind	Coalescing rule
chase on the same subjectKey	If a chase is QUEUED for project:X and a second stall signal arrives for project:X, merge the new signal into the pending request's trigger.signal[]. One chase, one message.
ic_reply on the same thread	Messages from one person within 20 s are merged into one run (people send in bursts). The run sees them as one turn.
replan on the same org	Multiple change events within 5 min produce one replan.
content_render	Renders for the same cadence occurrence are merged.

Coalescing happens only while the target is QUEUED or DEFERRED; a RUNNING run is never mutated. Coalesced requests emit run.coalesced{into: runId} and their idempotency keys are recorded against the surviving run.

Outreach-level coalescing (two Chonks messaging one person within 5 min) is separate and lives in outreach-service (Communication Engine Rules).

5. Retries and backoff

The executor classifies failures; the scheduler applies the table. Backoff is base × 2^attempt with full jitter, capped.

Failure class	Examples	Retryable	Max attempts	Base	Cap	Then
MODEL_TRANSIENT	429/5xx from provider, timeout	Yes	3	2 s	30 s	Gateway fallback provider is tried inside the call first; if all fail → FAILED_RETRYABLE
MODEL_OUTPUT_INVALID	Structured output failed schema after gateway retry	Yes, once, with the validation error appended to the step input	1	0	—	FAILED with class recorded; windowed → fallback
TOOL_TRANSIENT	Mirror query timeout, function service 503	Yes	3	1 s	15 s	FAILED_RETRYABLE
CONNECTOR_RATE_LIMITED	429 with Retry-After	Yes	5	Retry-After	15 min	Actuator writes wait; reads use stale mirror and continue
CONNECTOR_UNAVAILABLE	5xx, network	Yes	3	30 s	10 min	integration.connector.degraded after 3
CONNECTOR_AUTH	401/403, revoked token	No	0	—	—	FAILED; admin escalation; connector marked needs_reauth
POLICY_BLOCKED	Admission or action verdict BLOCKED	No	0	—	—	FAILED (admission) or action skipped (action gate); never retried
BUDGET_EXCEEDED	Any budget dimension	No	0	—	—	Fallback or dead-letter; see §7
ORG_INVARIANT	Org Model write rejected	No	0	—	—	Action fails; run continues; MoR escalation if the write was required by a cadence
LEASE_LOST	Executor crashed; lease expired; another worker resumed	Yes	unbounded (resume)	0	—	Resume from last run.step.completed
SURFACE_DELIVERY	Slack down, SMS undeliverable	Yes (in outreach-service)	5 over 30 min	30 s	10 min	Fallback surface per surfaceOrder; escalation messages try every surface
INTERNAL	Bug, unexpected exception	Yes	2	5 s	30 s	DEAD_LETTER; ops page

Dead-lettered runs are kept with full events and can be replayed after a fix. A dead-lettered windowed cadence run always produces a cadence.missed escalation; a dead-lettered chase produces a note in the next brief of the affected person ("I tried to chase X and couldn't; here is what I know").

6. Ordering guarantees

• Per (tenant, subjectKey): FIFO and exclusive. At most one run per subject is ADMITTED or RUNNING; queued runs on the subject start in request order. WAITING runs release the subject.
• Across subjects: no ordering. Two people's briefs run in any order.
• Events per tenant: totally ordered by seq (a Postgres sequence per tenant, assigned in the event insert transaction). Consumers (read models, audit, metrics) are idempotent on (tenantId, seq).
• Integration events: at-least-once, provider order not guaranteed. Mirrors are last-writer-wins on updatedAt; detectors evaluate the mirror state, not the event, so out-of-order deliveries converge.
• Human decisions: exactly-once effect. A decision resume is idempotent by dec:{id}:{status}; the actuator write uses the decision id as its provider idempotency key.
• Follow-ups chain on the parent's subject key, so a chase's 4-hour follow-up cannot run concurrently with a new chase on the same project.

Subject keys by run kind: briefs, check-ins, 1:1s, IC replies → person:{id}; chases and follow-ups → project:{id} (or person:{id} when the stall is a person's own item with no project); reviews, replans, schedule publish, huddles → org:{id}; owner update, quarterly planning, integrity checks → tenant.

7. Event sourcing and replay

7.1 Envelope

interface Event<T extends string = string, P = unknown> {
  id: EventId;
  tenantId: TenantId;
  seq: number;                    // per-tenant total order
  ts: ISODateTime;
  type: T;                        // catalog in doc 17
  payload: P;
  runId?: RunId;
  stepId?: string;
  correlationId: string;          // the root trigger's id; threads a chase and its follow-ups together
  causationId?: EventId;          // the event that caused this one
  actor: Actor;
  schemaVersion: string;
  hash: string;                   // sha256(prev.hash + canonical(this)) — same chain audit-service anchors
}

Events are written to Postgres (events table, partitioned by tenant and month) in the same transaction as any state they describe; an outbox relay publishes them to the stream for read models and audit. Nothing happens without an event and no event is written without its state.

7.2 Step events

A run emits: run.requested, run.queued, run.admitted, run.started, then per step run.step.started / run.step.completed{stepId, kind, inputHash, outputHash, usage, modelCalls: [{requestHash, provider, model, cachedTokens}]}, per action policy.evaluated{action, verdict} and one of action.executed | action.shadowed | decision.proposed | decision.recorded | policy.violation_blocked, then memory.fact.written[], then a terminal run.succeeded | run.failed | run.budget_exceeded | run.cancelled | run.dead_lettered. Model requests and responses are stored by requestHash in a content-addressed blob store (encrypted with the tenant key) so replay can substitute them.

7.3 Replay modes

type ReplayMode = 'VERIFY' | 'SHADOW_REEXEC' | 'LIVE_REEXEC';

Mode	Model calls	Side effects	Use
VERIFY	Recorded responses substituted by requestHash	None	Prove the harness is deterministic given the model; CI on every deploy replays a fixed corpus and diffs ProposedAction[] and verdicts.
SHADOW_REEXEC	Live, against the pinned Org Model version and recorded mirror snapshots	None; actions recorded as action.shadowed	Evaluate a new model, prompt, or policy against history.
LIVE_REEXEC	Live	Real	Admin-only, with a reason, for recovering a dead-lettered run after a fix. Idempotency of actuator writes still holds.

A replay run has its own runId and trigger.kind='REPLAY'; it references the source run. Replays never emit outreach unless LIVE_REEXEC.

8. Budgets

8.1 Per run

interface RunBudget {
  maxModelCalls: number;
  maxInputTokens: number;
  maxOutputTokens: number;
  maxWallMs: number;
  maxToolCalls: number;
  maxOutreach: number;            // messages this run may request; outreach-service enforces per-person caps separately
  maxCostUsd: number;
}

Defaults by run kind (owner-editable within the tier caps; the tier caps are Sea Suite ops config):

Run kind	Model calls	In tokens	Out tokens	Wall	Tool calls	Outreach	Cost USD
ic_reply	3	20k	1.5k	8 s (5 s p50 target)	6	1	0.06
morning_brief	4	30k	1.5k	60 s	12	1	0.10
checkin / one_on_one	3	25k	1.5k	60 s	8	1	0.08
chase / followup	4	40k	2k	120 s	15	3	0.15
approval	4	40k	2k	90 s	10	2	0.15
escalation	2	15k	1k	30 s	5	uncapped to MoR/owner	0.05
weekly_review	8	120k	6k	10 min	40	2	0.60
owner_update	12	250k	8k	15 min	60	1	1.50
quarterly_planning / replan	20	400k	16k	30 min	100	0 (decisions only)	3.00
schedule_publish	8	100k	4k	10 min	40	1 per scheduled person (deterministic notices)	0.50
shift_swap	3	20k	1k	45 s	8	4	0.06
content_render	2	20k	4k	20 min (render)	4	1	0.40 + render cost
onboarding_draft	30	600k	30k	60 min	300	0	6.00

On any dimension exceeding: the executor aborts the current step, emits run.budget_exceeded{dimension, usage}, and either delivers the run kind's deterministic fallback (brief, huddle notes, checklist prompt, text-only owner update from the org narrative) or dead-letters. Budgets are never silently raised by retries; each attempt gets the same budget.

8.2 Per tenant

maxDailyCostUsd = managedPeople × plan.perPersonDailyModelUsd (plan-defined; the constraint behind it is the PRD's "model spend well under 10% of price"). At 80%: P4/P5 runs deferred to the next day. At 100%: tiers downgraded (reasoning → standard → fast). At 150%: only P0–P2 run, with fallbacks. The Chief Fin Officer includes tenant model spend in the owner update when any threshold is hit.

8.3 Per person (outreach)

Enforced in outreach-service from outreach_cap policies: default 8 messages/day, 3 chases/day, quiet hours 19:00–07:00 local, one brief per day, coalescing window 5 min. Exempt: IC → human escalations, cadence.missed escalations, safety alerts, and call-out handling in atoms (callOutException). Suppressed messages emit outreach.suppressed{reason} and the run is told, so a chaser that cannot message escalates instead of retrying.

9. Policy gates

Three gates, three moments. The policy engine is pure; the gates are where it is called.

9.1 Admission gate (scheduler)

Checks, in order: tenant not paused; tenant daily budget not at 150% for this priority; Org Model invariants hold for the subject org (I-ORG-01/02 — otherwise the cadence is skipped with cadence.missed rather than run on a broken org); the run's function is not DISABLED for the tenant (Function mode); required connectors are at least degraded (a brief with no mirrors at all falls back). Output: functionModes for the run (see §9.4) pinned into the run.

9.2 Action gate (executor)

Every ProposedAction is classified: classify(action, ctx) → Verdict { mode, reasons: RuleEvaluation[], gates: string[], policyIds: PolicyId[] }. The action's declared class comes from the action catalog (Policy Engine §2); policies narrow it; graduation and phase cap it; HUMAN_ONLY is a floor that nothing lifts. The verdict is recorded in policy.evaluated before anything else happens.

9.3 Send gate (outreach-service)

Per-recipient caps, quiet hours, surface availability, tone bounds for the surface, symmetry (a message to the owner about a person is mirrored to that person's "what my manager knows" page), and the escalation exemption. A message that fails the send gate is returned to the run as OutreachResult{status:'suppressed', reason}.

9.4 Effective execution mode

type ExecutionMode = 'SHADOW' | 'PROPOSE_CONFIRM' | 'AUTONOMOUS' | 'HUMAN_ONLY' | 'BLOCKED';

function effectiveMode(declared: AutonomyClass, fn: ManagementFunction, ctx): ExecutionMode {
  if (declared === 'HUMAN_ONLY') return 'HUMAN_ONLY';                        // floor; nothing lifts it
  if (ctx.policyVerdict === 'BLOCKED') return 'BLOCKED';                      // hard line
  if (ctx.tenant.phase === 'LISTENING') {
    if (ctx.action.kind in LISTENING_ALLOWLIST) return 'AUTONOMOUS';           // intro message, questionnaire, read
    if (ctx.action.kind === 'PROPOSE_FIRST_PLAN') return 'PROPOSE_CONFIRM';
    return 'SHADOW';
  }
  const g = ctx.graduation[fn];                                               // 'SHADOW' | 'PROPOSE' | 'AUTONOMOUS'
  if (g === 'SHADOW') return 'SHADOW';
  if (g === 'PROPOSE') return 'PROPOSE_CONFIRM';
  return ctx.policyNarrowed ?? declared;                                      // AUTONOMOUS stays AUTONOMOUS unless a policy narrows it
}

Declared class \ function state	LISTENING	SHADOW	PROPOSE	AUTONOMOUS
AUTONOMOUS	SHADOW (allowlist: AUTONOMOUS)	SHADOW	PROPOSE_CONFIRM	AUTONOMOUS
PROPOSE_CONFIRM	SHADOW (first plan: PROPOSE_CONFIRM)	SHADOW	PROPOSE_CONFIRM	PROPOSE_CONFIRM
HUMAN_ONLY	HUMAN_ONLY	HUMAN_ONLY	HUMAN_ONLY	HUMAN_ONLY

SHADOW means the action is fully prepared, written to the shadow log with its receipt, and visible to the manager of record in admin for review, but nothing leaves the system. HUMAN_ONLY in any state prepares the packet and routes it; the packet is the product for those decisions.

10. Concurrency and backpressure

10.1 Limits

Scope	Default	Formula / cap
Per tenant concurrent runs	8	clamp(ceil(managedPeople / 10), 4, 64)
Per tenant per Chonk	4	Prevents one Chonk's planning burst from starving IC replies
Per subject	1	Exclusive lease
Per tenant queue depth	200	Above → L1 backpressure
Per cell executor pool	autoscaled	Target 70% utilization; scale on queue age, not depth
Model gateway per tier	tokens/min reservation per cell	Requests beyond reservation queue with priority ordering

10.2 Backpressure levels

Level	Trigger	Effect
L0	Normal	—
L1	Tenant queue > 100, or model p95 latency > 2× baseline	P4/P5 deferred (resume timer 15 min); follow-up timers stretched 1.5×
L2	Tenant queue > 200, or gateway queueing > 60 s	P3 deferred with deadline = now + 4 h; windowed P2 runs whose window closes within 10 min are pre-rendered with the fallback template
L3	Cell degraded (system.degraded), or tenant daily budget ≥ 150%	Only P0–P1 run; P2 delivered as fallback; everything else deferred; owner and admins notified

Deferral keeps the idempotency key and increments nothing; a deferred run is still one run. Levels are evaluated per tenant every scheduler tick and published as tenant.backpressure{level} on change.

10.3 Missed-run detection

For every windowed cadence occurrence, the scheduler tracks (cadenceId, participantId, fireAt) → satisfiedBy: runId?. At windowEnd + graceMinutes without a SUCCEEDED run, it emits cadence.missed and opens an escalation (kind=CADENCE_MISSED, severity by priority) routed to the org's manager of record and to Sea Suite ops. This is uncancellable and uncapped. A fallback delivery counts as satisfied but is separately reported as cadence.degraded.

11. Listening period and graduation

11.1 Listening period

Starts at listening.started when the owner approves the Org Model draft; ends 14 days later (listening.ended) or earlier only by owner action with a reason (never earlier than 5 days). During listening:

• All functions are effectively SHADOW (§9.4). Shadow runs execute fully; their outputs populate the admin shadow log so the owner can see what the managers would have done.
• The LISTENING_ALLOWLIST permits: reading every connected system; one intro message per person from their Chonk (listening_intro run kind; 1 outreach, plain tone ≤ 0.4); the async "meet everyone" questionnaire and replies to it; answering direct questions in-thread (ic_reply) with the disclaimer that the manager is not yet acting.
• The first plan is a PROPOSE_FIRST_PLAN decision to the owner (PROPOSE_CONFIRM), prepared by the Chief Operating Orca with Chonksworth's cover note.
• HUMAN_ONLY packets and IC → human escalations work exactly as in production.

11.2 Graduation state per function

type GraduationState = 'SHADOW' | 'PROPOSE' | 'AUTONOMOUS';

interface Graduation {
  tenantId: TenantId;
  fn: ManagementFunction;
  state: GraduationState;
  since: ISODateTime;
  ownerCap?: GraduationState;       // owner-set ceiling ("never above PROPOSE for budget")
  stats: { runs: number; proposals: number; confirmedUnedited: number; edited: number; rejected: number; overrides: number; incidents: number; windowStart: ISODateTime };
  history: Array<{ at: ISODateTime; from: GraduationState; to: GraduationState; reason: string; actor: Actor }>;
}

Transition	Criteria	Autonomy of the transition
SHADOW → PROPOSE	Listening ended; ≥ 10 shadow runs for the function; MoR shadow-review acceptance ≥ 80% over the last 10; no open incident for the function	PROPOSE_CONFIRM: the harness proposes, the owner taps
PROPOSE → AUTONOMOUS	≥ 20 proposals over ≥ 14 days; ≥ 90% confirmed without edit; rejected ≤ 5%; no incident in the window; not above ownerCap	PROPOSE_CONFIRM to the owner
Any → lower (regression)	Incident: a policy.violation_blocked for the function; an IC escalation naming the function; override rate > 20% over the last 10; two cadence.missed in 7 days for the function's cadences	AUTONOMOUS (the harness regresses immediately and notifies owner and MoR with the receipt); re-advancing requires the criteria again
Any → any (owner)	Owner narrows or widens scope	Human; recorded as a graduation_change decision with reason

Function mode deployments start the enabled functions in SHADOW like everyone else; disabled functions have no graduation record and their run kinds fail admission.

12. Scheduler loop

One leader per shard (a shard is a hash range of tenants in a cell); leadership via lease in Postgres; the durable state below is in Postgres with a Redis hot copy for the due-indexes.

STATE (durable):
  cadence_index : (tenant, cadenceId, participantId?) → { nextFireAt, windowEnd, grace, priority, chonkId, runKind, subjectKey, expirable }
  occurrences   : (tenant, cadenceId, participantId?, fireAt) → { satisfiedBy?: runId, missedMarked: bool, degraded: bool }
  timer_index   : (tenant, timerId) → { kind: FOLLOWUP | DECISION_EXPIRY | DECISION_REMINDER | DEFERRED_RESUME, dueAt, payload }
  queues        : per tenant: priority heap of RunRequest with per-subject FIFO lists
  leases        : (tenant, subjectKey) → { runId, expiresAt }
  idempotency   : (tenant, key) → runId, ttl 7d
  tenant_state  : tenant → { phase, paused, concurrencyLimit, running, backpressureLevel, dailySpendPct }

LOOP:
  while leader():
    now = clock.now()
    tick_cadences(now)
    tick_timers(now)
    detect_missed(now)
    for tenant in shard.tenants():
      tenant_state[tenant].backpressureLevel = compute_level(tenant, cell_health())
      admit(tenant, now)
    sleep(min(1s, time_until(next_due_anything())))

tick_cadences(now):
  for e in cadence_index.due(shard, now):                          # nextFireAt <= now
    if tenant_state[e.tenant].paused:
      emit(cadence.skipped{e, reason:'tenant_paused'})
    else:
      req = RunRequest{
        runId: new_id('run'), tenantId: e.tenant, chonkId: e.chonkId, runKind: e.runKind,
        trigger: CADENCE_TICK{cadenceId: e.cadenceId, participantId: e.participantId, fireAt: e.nextFireAt},
        idempotencyKey: key(e.tenant, 'cad', e.cadenceId, e.participantId ?? '-', minute(e.nextFireAt)),
        subjectKey: e.subjectKey, priority: e.priority,
        deadline: e.windowEnd, expirable: e.expirable, coalescable: false,
        budget: budget_for(e.runKind, e.tenant), requestedAt: now, attempt: 0 }
      occurrences.put(e.tenant, e.cadenceId, e.participantId, e.nextFireAt, {})
      enqueue(req)
    e.nextFireAt = rrule.next_after(e.schedule, e.nextFireAt, participant_tz(e))
    e.windowEnd  = window_end_for(e, e.nextFireAt)
    persist(e)

tick_timers(now):
  for t in timer_index.due(shard, now):
    switch t.kind:
      FOLLOWUP:          enqueue(RunRequest.from_followup(t.payload))        # key fup:{parentRunId}:{stage}; subjectKey = parent's
      DECISION_REMINDER: outreach.send(reminder(t.payload.decisionId))       # deterministic, to routedTo
      DECISION_EXPIRY:   decisions.expire(t.payload.decisionId)
                         enqueue(RunRequest.resume(t.payload.decisionId, status:'expired'))
      DEFERRED_RESUME:   queues.undefer(t.tenant, t.payload.runId)
    timer_index.delete(t)

detect_missed(now):
  for o in occurrences.overdue(shard, now):                        # windowEnd + grace < now, satisfiedBy == null, !missedMarked
    emit(cadence.missed{o})
    escalation_router.open(kind:'CADENCE_MISSED', tenant:o.tenant, scope:cadence(o).scope, severity: severity_for(cadence(o).priority), receipt: occurrence_receipt(o))
    o.missedMarked = true; persist(o)

enqueue(req):
  tx:
    if idempotency.exists(req.tenantId, req.idempotencyKey):
      emit(run.deduplicated{req.runId, original: idempotency.get(...)}); return
    idempotency.put(req.tenantId, req.idempotencyKey, req.runId)
    if req.coalescable and (target = queues.find_pending(req.tenantId, req.subjectKey, req.runKind)):
      queues.merge(target, req); emit(run.coalesced{req.runId, into: target.runId}); return
    queues.push(req.tenantId, req)
    emit(run.requested{req}); emit(run.queued{req.runId})

admit(tenant, now):
  ts = tenant_state[tenant]
  while ts.running < ts.concurrencyLimit:
    req = queues.next_eligible(tenant, ts.backpressureLevel, per_chonk_limit=4)   # highest priority; oldest first; skips subjects with a live lease; skips priorities excluded at this level
    if req == null: break
    if req.deadline and now > req.deadline and req.expirable:
      queues.drop(req); emit(run.expired{req.runId})
      if is_windowed_cadence(req): deliver_fallback(req); occurrences.mark(req, satisfiedBy: 'fallback', degraded: true)
      continue
    if not leases.acquire(tenant, req.subjectKey, req.runId, ttl: 2 * req.budget.maxWallMs):
      queues.requeue_tail(req); continue
    version = orgmodel.current_version(tenant)
    verdict = policy.pre_admission(req, version)
    if verdict.block:
      leases.release(tenant, req.subjectKey); queues.drop(req)
      emit(run.failed{req.runId, class:'POLICY_BLOCKED', reasons: verdict.reasons})
      if is_windowed_cadence(req): occurrences.mark(req, satisfiedBy: null)     # will surface as cadence.missed
      continue
    run = Run.from(req, orgModelVersion: version, functionModes: verdict.functionModes, leaseId: leases.id(...))
    ts.running += 1; persist(ts)
    emit(run.admitted{run})
    dispatch(run)                                                   # publish to executor stream, partitioned by tenant

Lease expiry: if an executor dies, the lease expires and the scheduler re-dispatches the same run with resume=true; the executor resumes from the last run.step.completed (LEASE_LOST in §5).

13. Run executor

EXECUTE(run):
  emit(run.started{run.runId, attempt: run.attempt, resume: run.resume})
  meter = BudgetMeter(run.budget)                                   # throws BudgetExceeded on any dimension
  try:
    ctx = assemble_context(run)
      # ctx = { orgModel: orgmodel.read(run.tenantId, asOf: run.orgModelVersion, scope: scope_for(run)),
      #         memory: memory.read(subjects_for(run), scope: run.runKind, reader: run.chonkId),
      #         mirrors: mirrors.snapshot(run.tenantId, sources_for(run)),      # returns data + staleSince per source
      #         policies: orgmodel.policies_for(run),
      #         charter: charters[run.chonkId], voice: voice_for(run.tenantId, run.chonkId),
      #         functionModes: run.functionModes, budget: run.budget, receipts: ReceiptBuilder(run) }

    plan = step(run, 'planner', () => chonk.planner(ctx, objective_for(run)))       # StepPlan: steps[], expectedActions[]
    assert plan.steps.every(s => s.tools ⊆ ctx.charter.toolAllowlist)              # else FAILED{class:'INTERNAL'} — planner asked for a tool it cannot have
    assert plan.steps.length <= run.budget.maxToolCalls

    findings = []; drafts = []
    for s in plan.steps:
      meter.check()
      switch s.kind:
        'analyst': findings += step(run, s.id, () => chonk.analyst(ctx, s, tools: read_tools(ctx) ∪ function_services(ctx)))
        'chaser':  findings += step(run, s.id, () => chonk.chaser(ctx, s, findings))         # returns ProposedAction[] of SEND_MESSAGE / SCHEDULE_FOLLOWUP / ESCALATE / HANDOFF; sends nothing
        'writer':  d = step(run, s.id, () => chonk.writer(ctx, s, findings))
                   grounding = validate_grounding(d, ctx.receipts)                            # every [n] resolves; no uncited numeral, date, or name; length within surface limit
                   if not grounding.ok: d = step(run, s.id + ':regen', () => chonk.writer(ctx, s, findings, feedback: grounding.errors))
                   if not validate_grounding(d, ctx.receipts).ok: throw Failed('MODEL_OUTPUT_INVALID')
                   drafts += d
      # tool calls inside a step go through tool_gateway(run) which: checks allowlist, checks tenant scope, meters, records tool.called events, and returns mirror data with snapshot ids for receipts

    actions = plan.materialize_actions(findings, drafts)             # ProposedAction[]; each carries receiptId
    for a in actions:
      verdict = policy.classify(a, ctx)                              # §9.2 / §9.4
      emit(policy.evaluated{a, verdict})
      switch verdict.mode:
        'BLOCKED':         emit(policy.violation_blocked{a, verdict}); maybe_regress(run, a); continue
        'SHADOW':          shadow_log.write(run, a, verdict); emit(action.shadowed{a, verdict}); continue
        'AUTONOMOUS':      execute_action(run, a, verdict)
        'PROPOSE_CONFIRM': d = decisions.create(run, a, verdict)      # status 'proposed', expiresAt per decisionKind
                           outreach.send(approval_request(d, ctx))    # to routedTo per approval_route policy; send gate applies
                           timers.add(DECISION_REMINDER, d.expiresAt - 48h); timers.add(DECISION_EXPIRY, d.expiresAt)
                           run.pendingDecisions += d.id; emit(decision.proposed{d})
        'HUMAN_ONLY':      p = packets.create(run, a, verdict)        # Decision with status 'recorded' once the human answers; never executed by us
                           escalation_router.route(p)                  # to the human(s) the policy names; uncapped
                           emit(decision.recorded{p, stage:'packet_sent'})

    memory.commit(run, allowed_facts(findings, ctx))                 # write rules in doc 04 §6; emits memory.fact.written[]
    if run.pendingDecisions.nonEmpty:
      transition(run, 'WAITING_HUMAN'); leases.release(run.tenantId, run.subjectKey); return
    transition(run, 'SUCCEEDED'); emit(run.succeeded{usage: meter.usage})

  catch BudgetExceeded as e:
    transition(run, 'BUDGET_EXCEEDED'); emit(run.budget_exceeded{dimension: e.dimension, usage: meter.usage})
    if has_fallback(run.runKind): deliver_fallback(run); transition(run, 'SUCCEEDED', degraded: true)
    else: transition(run, 'DEAD_LETTER'); escalate_dead_letter(run)
  catch Retryable as e:
    transition(run, 'FAILED_RETRYABLE'); scheduler.schedule_retry(run, e.class)    # §5
  catch Exception as e:
    transition(run, 'FAILED'); emit(run.failed{class: classify(e)})
    if is_windowed_cadence(run): occurrences.mark(run, satisfiedBy: null)            # → cadence.missed via scheduler
  finally:
    leases.release_if_held(run.tenantId, run.subjectKey, run.leaseId)
    tenant_state.decrement_running(run.tenantId)

step(run, stepId, fn):
  if run.resume and events.has_completed(run.runId, stepId): return events.result(run.runId, stepId)   # resume after LEASE_LOST
  emit(run.step.started{stepId})
  result = run.replay?.mode == 'VERIFY' ? replay.substitute(run, stepId, fn) : fn()
  emit(run.step.completed{stepId, outputHash: hash(result), usage: meter.delta()})
  return result

execute_action(run, a, verdict):
  switch a.kind:
    'SEND_MESSAGE':      r = outreach.send(OutreachRequest.from(a, run, verdict))   # send gate; may return 'suppressed'
                         if r.status == 'suppressed': run.notes += r; emit(outreach.suppressed{a, r.reason})
    'ORG_CHANGE':        orgmodel.apply(Change{ops: a.ops, reason: a.reason, actor: {kind:'run', id: run.runId, onBehalfOf: verdict.decidedBy}})
    'CONNECTOR_WRITE':   connectors.actuate(a.provider, a.op, a.args, idempotencyKey: run.runId + ':' + a.id)
    'SCHEDULE_FOLLOWUP': timers.add(FOLLOWUP{parentRunId: run.runId, stage: a.stage, dueAt: a.dueAt, subjectKey: run.subjectKey})
    'HANDOFF':           h = handoffs.create(run, a); scheduler.enqueue(RunRequest.from_handoff(h))
    'ESCALATE':          escalation_router.open(a.escalation)
    'RENDER_CONTENT':    content.render(a.job) → callback → outreach.send(...)    # run enters WAITING_EXTERNAL if the send must be in this run
    'MEMORY_WRITE':      (never direct; memory.commit handles it)
  emit(action.executed{a, verdict, receiptId: a.receiptId}); audit.append(run, a, verdict)

RESUME(run, decision):                                              # trigger HUMAN_DECISION
  a = run.pendingActions[decision.id]
  ctx = assemble_context(run, orgModelVersion: orgmodel.current_version(run.tenantId))    # current, not pinned
  switch decision.status:
    'confirmed':
      v2 = policy.classify(a, ctx)
      if v2.mode in ['AUTONOMOUS', 'PROPOSE_CONFIRM']: execute_action(run, a with {decidedBy: decision.decidedBy}, v2); emit(decision.executed{decision})
      else: decisions.mark_stale(decision, v2); outreach.send(stale_notice(decision)); emit(decision.stale{decision, v2})
    'rejected':  emit(decision.rejected{decision}); outreach.send(rejection_notice(decision)); memory.commit(run, [rejection_fact(decision)])
    'expired':   emit(decision.expired{decision}); outreach.send(expiry_notice(decision))
  run.pendingDecisions -= decision.id
  if run.pendingDecisions.isEmpty: transition(run, 'SUCCEEDED')

maybe_regress implements §11.2's incident rule: a BLOCKED verdict on a function in AUTONOMOUS regresses it to PROPOSE in the same transaction that records the block.

14. Risk controls (closed loops)

RC-HARN-01 — Silent cadence skip. Owner: Chief Operating Orca. Metric: cadence.missed without an escalation delivered within 15 min (target 0); occurrences with no terminal run (target 0).

• Detect: occurrences sweep in detect_missed; a separate watchdog (outside the scheduler) counts occurrences per day versus expected from the cadence index and alarms on a shortfall.
• Contain: escalation to MoR and ops; fallback delivery for briefs and huddles.
• Correct: replay the missed run (LIVE_REEXEC if the content is still useful, else skip with a note in the next occurrence).
• Learn: every missed occurrence gets a failure class; classes trending up are a scheduler bug or a budget mis-sizing, and the fix ships with a regression fixture in the replay corpus.

RC-HARN-02 — Chase loops. Owner: Chief Build Officer. Metric: follow-up depth per correlation id (target ≤ 3); outreach suppressed per person per week (target < 2).

• Detect: FOLLOWUP.stage > 3 on one correlation id; suppression counts in outreach-service; the same recipient chased by two correlation ids for the same external ref.
• Contain: stage cap (a follow-up beyond stage 3 becomes an escalation, never another message); per-person chase cap.
• Correct: the chaser's follow-up ladder is reviewed for the signal kind; the detector's re-fire suppression window is widened.
• Learn: the ladder is a policy (approval_route/outreach_cap) rather than prompt text, so the fix is a policy change with a reason and a test in sim.

RC-HARN-03 — Replay divergence. Owner: Chief Security Officer. Metric: VERIFY replay diff rate on the CI corpus (target 0).

• Detect: nightly VERIFY replay of a fixed corpus of 200 runs across kinds; any diff in ProposedAction[] or verdicts fails the build.
• Contain: the deploy is blocked.
• Correct: the diverging step is diffed (context assembly, tool gateway, policy engine); nondeterminism is removed or the recorded input is extended.
• Learn: the diverging run joins the corpus permanently.

15. Edge cases

ID	Scenario	Signal / detection	Rule	Action (autonomy class)	Escalation	Test
EC-HARN-001	Scheduler leader fails over during tick_cadences; the new leader sees the same due entries	Idempotency key cad:…:{minute} already present	Cadence firings are idempotent per minute	run.deduplicated (AUTONOMOUS)	None	Kill the leader mid-tick in sim; assert exactly one run per occurrence
EC-HARN-002	A person's brief window is 08:00–08:30 and the tenant is at L2 backpressure at 08:22	Windowed P2 with window closing within 10 min at L2	Briefs never silently skip	Pre-render and deliver the fallback brief; mark occurrence degraded (AUTONOMOUS)	cadence.degraded to ops; no MoR escalation	Force L2 at 08:22 in sim-lanternfish; assert fallback delivered by 08:30
EC-HARN-003	Owner narrows budget function to PROPOSE while a chase run is RUNNING with an AUTONOMOUS functionModes snapshot	Graduation change event during run	Admission snapshot governs the current run; the action gate re-reads graduation at action time	Action re-classified at action time → PROPOSE_CONFIRM (AUTONOMOUS re-classification)	None	Change graduation mid-run in sim; assert a decision was created, not an execution
EC-HARN-004	Executor crashes after outreach.send returned but before action.executed was written	Resume finds outreach.sent event for the action's receipt but no action.executed	Outreach-service is idempotent on (runId, actionId)	Re-issue send; outreach-service returns the prior delivery; write action.executed (AUTONOMOUS)	None	Kill the worker between the two writes; assert one message delivered
EC-HARN-005	IC replies to a brief at 23:40 local ("are you there?")	IC_MESSAGE during recipient quiet hours	Quiet hours cap outbound initiated messages; replies to a person's own message are allowed within 10 min	Reply (AUTONOMOUS); no new chases initiated	None	Send a message at 23:40 in sim; assert reply and no chases
EC-HARN-006	Two HUMAN_DECISION triggers for one decision: owner approves on phone and in the console within a second	Second key dec:{id}:confirmed deduplicated	Exactly-once effect	run.deduplicated (AUTONOMOUS)	None	Double-post approval; assert single actuator write
EC-HARN-007	A follow-up fires for a project whose PR merged 10 minutes ago (mirror lagged)	Chaser's analyst step reads the mirror and sees merged	Detectors evaluate state, not events	Close the correlation with signal.stall.cleared; send a one-line "resolved, thanks" only if a prior chase went out (AUTONOMOUS)	None	Merge the PR in the mirror before the follow-up; assert no chase
EC-HARN-008	Tenant daily model budget hits 150% at 14:00	tenant_state.dailySpendPct ≥ 150	L3 for the tenant	P0–P1 continue; P2 fallbacks; owner and admin notified; Chief Fin Officer adds a line to the owner update (AUTONOMOUS)	Owner (informational)	Set a tiny budget in sim; assert level and notifications
EC-HARN-009	A replay in SHADOW_REEXEC proposes a HUMAN_ONLY packet	Verdict HUMAN_ONLY inside a replay	Replays never emit outreach	Packet written to the shadow log only (SHADOW)	None	Replay a run whose source produced a packet; assert no escalation opened
EC-HARN-010	Onboarding draft run exceeds 60-minute wall budget on a 180-person tenant	BUDGET_EXCEEDED{dimension:'wall'}	Drafts are resumable per connector	Persist partial draft; enqueue continuation with parentRunId and the remaining connectors (AUTONOMOUS)	Admin sees "draft 60% complete"	Seed 180 people with slow mirrors; assert continuation and a complete draft
EC-HARN-011	Listening period: an IC writes "I want to report a safety issue" on day 3	Deterministic intent list in ingress	HUMAN_ONLY and escalation work during listening	Packet to the org's MoR and the designated safety contact over every surface (HUMAN_ONLY, uncapped)	MoR; owner if unacknowledged in 1 h	Send the phrase during listening in sim-harbortable; assert delivery and no autonomous action
EC-HARN-012	The same GitHub PR emits review_requested then review_request_removed within a second, out of order	Mirror last-writer-wins on updatedAt; detector evaluates mirror state	Out-of-order delivery converges	No stall signal (AUTONOMOUS)	None	Deliver events reversed; assert no chase

Open questions

1. Leases in Postgres versus Redis. Postgres leases are simpler to reason about with the event transaction; Redis is faster. Start with Postgres advisory locks and measure at 25k people per cell.
2. Per-person concurrency for atoms. A call-out at shift start can generate a swap run, a brief, and a huddle for the same person within minutes; the person: subject serialization may add unacceptable latency. Consider a swap fast lane that bypasses the subject lease but still respects outreach caps.
3. Budget units. Cost in USD ties budgets to provider pricing; a normalized "compute unit" would survive provider changes. Decide before the tier mapping becomes editable by customers.
4. Coalescing IC messages at 20 s. Too short for people who type in fragments, too long for the 5 s reply target. Instrument in pilots.
5. Graduation criteria numbers. 10 / 80% / 20 / 90% are starting points; the concierge pilots should tell us where the trust cliff actually is.

Related docs

• System Architecture
• Org Model Spec
• Manager Agents
• Policy Engine
• Decision Pipeline
• Closed-Loop Risk Mitigation
• Edge Case Catalog
• Escalation and Human-in-the-Loop
• Communication Engine Rules
• Evaluation and Testing
• Observability and Runbooks
• Rollout and Operations
• Data Model and API Reference