LLM-at-Joints Fleet Drift (2026-04-22 Audit)

LLM-at-Joints Fleet Drift (2026-04-22 Audit)

Context

Operator directive 2026-04-22: "we have drifted in common sense ways in the past. Especially by neglecting to put intelligence in the seams (e.g. instead of regex haha etc. you know what I mean, metaphorically)."

A 15-agent parallel audit ran across the camber + heartwood + orbit fleet on 2026-04-22T19:04Z, covering nine LLM-in-the-seams seam audits and six big-picture drift audits. This observation records the findings as one connected pattern.

Observation

The fleet has shipped the "intelligence in the seams" library (CMB-0174, 2026-04-21) but has not activated it.

Quantified state:

Library shipped. Schema missing. Consumers not ported. BHAG silent. Provider-neutrality undone. Rules still running where intelligence belongs.

Evidence (9 seam audits + 6 drift audits)

Seams still running rules where LLM-primitive-call belongs

1. Vendor resolution (gmail-financial-pipeline/extraction.ts::extractVendor) — 100% rules-based. Substring match against vendor_registry Map + from-header regex. Unknown-vendor silent fallthrough to from-header display name. match() primitive idle, never called. 2. Cost-code attribution — suggestReceiptCostCode() DOES use resolve() primitive (this is the one bright spot). But underlying vendor→cost lookup is hardcoded static JSON Map (Woodbery + Moss), no DB-backed lookup for 8 other active projects. 3. Project attribution (call path) — process-call/index.ts runs 6-tier heuristic ranking with no LLM tie-break at final step. Weak-evidence contacts capped at 0.35 confidence; when top two scores diverge <10 points, pure heuristic decides. resolve() primitive idle. 4. Classification + doc-type — Gmail v3 branch delegates to classify() primitive (correct path); photo pipeline has NO doc-type check (receipts and jobsite photos handled identically); SMS/Beside pipeline has ZERO classification (all messages treated uniformly regardless of intent). 5. Deduplication — 5 seams, all SHA256 of exact bytes / JSON. Photo source_ref dedup breaks when same photo attached to multiple BT folders OR BT re-encodes. Financial claim evidence_fingerprint breaks on JSON key-order drift. match() primitive idle. 6. Photo room attribution — cascade is token-first then vision-fallback. Filenames are epoch-ms (zero signal). The scorer "falls through to LLM correctly by accident." vision() primitive implemented but called directly against OpenAI, not via primitive router. 7. SMS/message content extraction — SMSBubble.swift:170 regex-scrapes message.content for URL patterns with hardcoded /service-message/medias/ path. Literal "regex haha" — Chad's named example. 8. Beside contact → project attribution — NO project_id column on beside_thread_events. Contact resolution via phone match but no project attribution path at all today. Path D (media_message promotion) will ship without this middle tier. 9. Schema drift detection — no detection tooling exists anywhere. CAMBER-02's 2026-04-22T16:45Z live-patch to project_change_events_event_type_check constraint was only caught because Gemini Winship verification manually asked about it. Without the human check, drift would proliferate silently.

Big-picture drift

10. Telos tracking — Recent 20 dispatches: T1 (zero manual finance) ~40%, T2 (client visual timeline) ~90% shipping, T3 (adversarial-synthesis BHAG) 0% — zero tickets, T4 (three-domain separation) ~20%. T3 has shipped orbit orbit_v2_get_brier_scores infrastructure but no consumer. 11. Three-domain separation — clean. 5 bridge libraries in heartwood→camber direction, all actively called, no dormant bridges. No camber→heartwood or orbit-bleed detected. Only structural boundary the fleet is holding. 12. Single-project hardcodes — 5 CRITICAL blockers: dollhouse-config.js only Woodbery, bill_cover.ts:141,866 hardcoded stylesheet URL, dollhouse-public-config.js:58 fallback slug, financial_project_cost_registry.ts:10-13 static imports, heartwood-finance-reconciliation-dashboard.js:1588 hardcoded logo path. Projects table is SSOT; consumers forgot to read it. 13. Provider neutrality — 15 functions hardcode OpenAI endpoint + model. Gemini adapter not present in primitive library router. BHAG architecturally blocked until retrofit. 14. Primitive library adoption — 3.6% of functions use it. Backing DB tables don't exist. Library is orphaned from its persistence layer. 15. E2E readiness — photo pipeline from BT scrape → Redline Mac is GREEN end-to-end for Winship (CAMBER-02 proved 2026-04-22T16:45Z). Weekly email digest is RED (no outbound email infra exists on HCB today).

Interpretation

This is not a regression. This is ORA-2026-0036 Dormant Capability pattern at its most concentrated: CMB-0174 shipped a 1,100+ LOC library with 6 primitives, a decision audit log, provider routing, caching — and the activation migration never landed, the 15 existing OpenAI-direct functions were never retrofitted, and the adversarial-synthesis BHAG that the library unlocks has zero tickets in flight.

Every single one of the 9 seam audits independently proposed the same fix: call one of the shipped primitives. Five agents converged on match(). Three on resolve(). Two on classify(). One on vision(). One on extract().

The drift is structural, not random. The fleet rewards shipping libraries (clean DONE posts with green tests). The fleet has no incentive shape for the cross-repo cross-file "retrofit 15 consumers and prove Brier-score superiority" work class. That work class lost to the library work class because library work is legible and activation work is messy.

This is the pattern ORA-2026-0036 named as doctrine on 2026-04-22. This observation records the instance that motivated the doctrine.

Countermeasure (program, 2026-04-22 wave)

Filed as a 7-ticket wave same day:

1. HWD-0288 (CRITICAL) — ship llm_joint_decisions + llm_joint_config migration to camber DB. Foundational unblock. 2. CMB-0190 (HIGH) — batch retrofit 15 OpenAI-direct functions to primitive library. Unblocks Gemini + BHAG. 3. CMB-0191 (HIGH) — retrofit 5 regex extraction seams in extraction.ts to extract() primitive. 4. CMB-0192 (NORMAL) — flip photo-room cascade from token-first to content-first via vision() primitive. 5. FLT-0249 (HIGH) — file the first adversarial-synthesis BHAG ship ticket (pick one joint, run parallel Claude/GPT/Gemini, compute Brier). Break the T3 0-count drift. 6. CMB-0193 (NORMAL) — semantic dedup via match() for photo source_ref collisions. 7. FLT-next (HIGH) — 5 single-project hardcodes batch (re-filed as FLT for cross-lane). 8. HWD-0290 (NORMAL) — schema drift daily watcher (catch CAMBER-02-class live patches before they proliferate).

Sequencing: HWD-0288 blocks everything else. Once migration lands, CMB-0190 retrofit wave fans out; seam tickets (CMB-0191, 0192, 0193) can parallel-claim. FLT-0249 BHAG ship is parallel to retrofit.

Maturity rationale

M2 rather than M1: audited across 15 independent agents + confirmed at multiple independent grep paths + proven by live DB schema check + measured with quantitative coverage metric (3.6% adoption). The pattern is observed, corroborated, and quantified.

Elevation gate

OBS M2 → HYP M3 transition requires: demonstrate that the 7-ticket remediation wave actually moves the metrics. After HWD-0288 + CMB-0190 ship: re-run the adoption-rate measurement. If coverage rises to ≥50% AND the first BHAG ship produces a result (pass or fail), elevate.

See also

• ORA-2026-0036 (parent: Dormant Capability)
• ORA-2026-0029 (dying-well FOLLOW-ONS: activation-ticket rule enforces the pattern prospectively)
• ~/Desktop/fleet/ARCHITECTURE_LLM_AT_JOINTS.md (ORA-02, 2026-04-18 — the design this observation measures)
• Agent outputs: /private/tmp/claude-501/-Users-chadbarlow/461214e0-*/tasks/ (15 audit transcripts)