The Failure Factory
An Agentic Aberrancy Atlas Shrugged
Follow the incidents, the machinery, and the human work holding the relationships together.
Enter the chapter8. The Failure Factory
The natural response to unreliability is control.
Add a check.
Add a guardrail.
Add a validator.
Add a rollback.
Add a retention policy.
Add a promotion gate.
Add a baseline.
Add a checkpoint.
Add a reservation.
Add a fairness counter.
Add a stagnation detector.
Add a history record so the next agent remembers why all the other controls exist.
We did exactly that.
Many of those controls made the system safer in the narrow place where the last failure had hurt us. They also created an entire second application made of supervision. That application had state, identity, readers, writers, migrations, ordering rules, resource costs, cleanup rules, authority relationships, and failure modes of its own.
Once that happened, every reliability mechanism became capable of generating the next reliability incident.
That is the Failure Factory.
8.1 Rollback protected the learner by rewriting the teacher's memory
Candidate rejection is one of the most sensible safety controls in the trainer.
A teaching visit creates a temporary version of the learner, applies one or more updates, and evaluates the result. If the candidate harms protected behavior or fails to create useful improvement, the trainer throws away that candidate and returns to the retained parent.
That keeps a bad update from becoming the next continuing learner.
One implementation also restored the teacher's unresolved-miss exposure counters from the pre-visit receipt. Those counters recorded which difficult examples had actually been delivered so later scheduling could distribute attention intelligently. [C15][GH02]C15 · Candidate rollback erased teacher observation historyFailure Atlas deep caseCandidate rejection correctly rolled back weights and incorrectly rolled back the teacher's memory of what it had just taught.GH02 · C15. Candidate rollback erased teacher observation historyGitHub issue / pull-request evidenceCandidate rejection correctly rolled back weights and incorrectly rolled back the teacher's memory of what it had just taught.
The candidate failed, so its weights disappeared.
The lesson had still happened.
The rollback erased the teacher's record of that fact.
Future scheduling could now select from a history that claimed the example remained underexposed, even though the system had just taught it.
The learner became safer.
The teacher became less honest.
This is a recurring Failure Factory pattern: a control built for one state domain gets applied to another state domain merely because the two participate in the same transaction.
Rollback makes sense for candidate model state.
Historical observation is an event record.
Treating them as one object converts safety into amnesia.
8.2 Stagnation protection became capable of enforcing stagnation
The project also needed a way to stop wasting compute on capabilities that showed no durable progress.
Early logic could quarantine a capability after only a couple of rejected recovery visits even when recent gains showed the learner was still moving. We repaired that with a capability-local no-progress floor and gain-based reset. Bounded live runs then completed with gains and avoided premature quarantine. [C23][D06]C23 · Local stagnation containment became a global stopFailure Atlas deep caseA local stagnation guard became a global stop button.D06 · C08. Rhetorical agreement can conceal a changed questionSession minutes / durable run documentAgreement became a camouflage layer.
That control worked at its intended layer.
The next long run exposed the next layer.
After roughly 2,340 seconds and 347 visits, local unresolved capabilities drove a global STAGNATION_REVIEW_REQUIRED stop while other developmental work remained available. [E:PM-2026-10-21-005]E:PM-2026-10-21-005 · C23. Local stagnation containment became a global stopEntif postmortem / incident recordA local stagnation guard became a global stop button.
The system had learned to ask a better local question: has this capability actually been stuck long enough to justify containment?
It had not learned the next question: should this local result end the entire run?
A mechanism designed to keep the project from wasting time on one dead end gained enough authority to block every other road.
That is how a sensible guardrail becomes a traffic jam.
8.3 Every state migration creates a migration for its readers
The content-addressed state cutover showed how quickly “we migrated the state” can become fiction.
The writer moved the logical school record behind content-addressed storage. state.json became a small pointer to the current logical state. That was the new representation.
One dry-run consumer still opened the pointer and tried to decode it as the old complete record. It crashed.
We repaired that reader.
The next real startup reached the dashboard and found another consumer with the same obsolete assumption. That path failed on a missing field and fell into an import of a module that had already been removed. [C25][D05]C25 · State serialization changed, but consumers still read the old representationFailure Atlas deep caseThe state migration succeeded at the producer and failed at the consumers.D05 · C17. Archive quality, candidate acceptance and protection policy divergedSession minutes / durable run documentThe system confused saving a better witness with protecting the learner that would continue training.
The migration was complete in the writer and incomplete in the system.
That distinction is especially dangerous in AI-assisted coding because the changed files are visible, the patch can be large, the tests around the patch can pass, and the agent can produce a beautiful architectural summary while old readers remain scattered across the application.
The implementation looks broad because a lot of code moved.
The semantic boundary remains open because one consumer still believes yesterday's world.
8.4 The direct Boolean repair sprawled before it closed its core boundary
The direct-U7C work supplied the feature-scale version.
The request was concrete: bring the historical direct Boolean teaching path into the current Grade B system.
The initial implementation expanded into broader runtime and migration infrastructure before the core replacement had been proven. After scope correction, the accepted implementation still exposed several boundary mistakes.
The same content digest was used both for a source specimen and for the curriculum-row role in which that specimen appeared. One Boolean expression could legitimately participate in more than one capability, so the state ended up with 802 teacher-miss rows but only 801 unique row identifiers. The admitted curriculum profile also retained a reduced roster even though the provider supported the full capability set. [C26][E:PM-2026-10-23-003][E:PM-2026-10-23-004]C26 · The direct-U7C cutover was incomplete at three different boundariesFailure Atlas deep caseThe direct-U7C cutover initially sprawled beyond the requested core change and still missed three real boundaries: role identity, capability roster, and downstream replay/retention.E:PM-2026-10-23-003 · C26. The direct-U7C cutover was incomplete at three different boundariesEntif postmortem / incident recordThe direct-U7C cutover initially sprawled beyond the requested core change and still missed three real boundaries: role identity, capability roster, and downstream replay/retention.E:PM-2026-10-23-004 · C26. The direct-U7C cutover was incomplete at three different boundariesEntif postmortem / incident recordThe direct-U7C cutover initially sprawled beyond the requested core change and still missed three real boundaries: role identity, capability roster, and downstream replay/retention.
The agent had done a lot of architecture.
The invariant that motivated the work was still leaking at multiple edges.
This became another recurring pattern: capable agents make architectural sprawl cheap. They can change many layers fast enough that breadth starts to feel like progress. The human later discovers that semantic closure is slower than code generation and the original hole is still sitting there, now surrounded by nicer abstractions.
8.5 “Reserved” capacity that can be spent first is performance art
The next failure arrived with arithmetic simple enough to explain on a napkin.
The trainer had four lesson slots.
Two mandatory preservation examples needed inclusion because earlier fragile behavior had to survive the new lesson.
An optional fresh family consumed three slots before the mandatory injector ran.
The injector arrived needing two slots and found one replaceable position.
The trainer stopped with FRAGILE_PRESERVATION_BATCH_CAPACITY_EXHAUSTED. [C27][E:PM-2026-10-23-005]C27 · Reserved batch capacity was only a labelFailure Atlas deep caseReserved capacity existed only after lower-priority work had already spent it.E:PM-2026-10-23-005 · C21. Mean-stability promotion changed the unit of measurementEntif postmortem / incident recordPromotion logic changed the unit of measurement midstream.
The code described some capacity as reserved.
The allocation order allowed optional work to spend it first.
Four chairs. Three optional guests. Two mandatory guests. Five bodies.
The control plane had enough abstraction to obscure furniture mathematics.
This is the kind of bug that looks absurd after discovery because the policy language sounds stronger than the executable ordering. “Reserved” is a social promise. Software needs a resource boundary.
8.6 A side quest acquired authority over the main quest
Championship fan-out later repeated the same inversion at workflow scale.
A legitimate title takeover created an opportunity to compare related capability owners. The scheduler expanded into sibling capabilities based on active incumbent titles. Some siblings had no comparable challenger evidence in the current run.
The generic cohort constructor received an empty comparison and correctly threw COHORT_EMPTY.
The whole Grade C command stopped. [C28][B:M125]C28 · Optional work escalated into a fatal invariantFailure Atlas deep caseOptional championship fan-out became a fatal invariant.B:M125 · C28. Optional work escalated into a fatal invariantBilqis Mistakes ledger recordOptional championship fan-out became a fatal invariant.
The constructor was strict for a good reason.
The caller had promoted optional work into a fatal requirement before checking whether the evidence needed for that work existed.
The main developmental path could have continued. A side opportunity lacked proof. The side opportunity killed the command.
This is what happens when priority semantics disappear inside generic invariants. Advisory, defensive, opportunistic, and mandatory work all begin with reasonable purposes. Once they flow through the same fatal boundary, their original rank no longer matters.
The system protects the side quest with the life of the main quest.
8.7 Retention policy created the storage crisis it was supposed to prevent
The soup store turned the Failure Factory into architecture satire.
The active policy intended to keep a small set of advisory model mixtures: current and immediately previous soups for the curriculum grades.
A mentor-epoch exemption bypassed retirement.
The managed store reached 925 soup files.
Cleanup later retired 919, representing roughly 39.5 billion logical bytes, and retained six current or previous A/B/C soups. Immediate checks showed stable manifest identity, student SHA, and optimizer step. [C29][D04]C29 · Retention cleanup and custody verification disagreed about what had to surviveFailure Atlas deep caseCleanup and custody implemented incompatible definitions of liveness.D04 · C14. A negative incentive that lost to the selection scaleSession minutes / durable run documentThe promised adaptive penalty existed numerically but lost by an order of magnitude to the scoring forces around it, while novelty kept minting functionally similar strategies under fresh identities.
Then the next real training retry failed because a different custody verifier still required 22 of the retired archived soups.
One safety system said the files were dead.
Another safety system said the application was unsafe without them.
Both had receipts.
This is the exact moment reliability engineering starts resembling theology: two internally coherent doctrines, both claiming authority over the same artifacts, and a human in the middle trying to get the damn trainer to boot.
The retention lifecycle had created the storage crisis it was meant to contain.
8.8 The slow learner was sometimes a fast learner trapped inside a slow government
The performance investigation exposed the resource cost of all that supervision.
From the user's seat, training often ran around 140 to 160 optimizer steps per minute, with occasional higher bursts and better historical rates. That looked like a slow neural update loop.
A bounded clone later measured the warm inner loop around 968 to 1,002 attempted steps per minute. [C30][D04]C30 · A slow control plane was mistaken for an inadequate storage engineFailure Atlas deep caseThe slow path sat in the orchestration surrounding the inner loop.D04 · C14. A negative incentive that lost to the selection scaleSession minutes / durable run documentThe promised adaptive penalty existed numerically but lost by an order of magnitude to the scoring forces around it, while novelty kept minting functionally similar strategies under fresh identities.
The math could move.
The bureaucracy around the math was eating the clock.
One startup path rehydrated 832 historical observations. A broad cold materialization ran for 71.95 seconds before interruption without finishing. A selective field read completed in 11.722 seconds.
The application had accumulated enough history assembly, policy selection, evidence construction, diagnostics, state loading, and between-visit work that the model looked slow from the outside.
We had built a sports car and required it to visit the DMV between laps.
This scales directly to agentic products. Vendors love to discuss model latency because model latency is measurable and glamorous. Users wait on orchestration, memory retrieval, tool calls, policy checks, state reconstruction, retries, validation, and all the other little governments surrounding the model.
The intelligence can be fast while the product moves like wet cement.
8.9 Observation joined the thing being observed
Trying to inspect one of those slow paths exposed another control tax.
A debugger attached to a live trainer that was consuming CPU while its visible status looked stale. The attach hung. The wrapper was interrupted. Afterward the trainer was gone and no normal STOP receipt existed. The persisted checkpoint survived. [C31][E:PM-2026-10-22-002]C31 · Observation was allowed to interfere with the system being observedFailure Atlas deep caseThe act of observing the trainer became capable of disturbing it.E:PM-2026-10-22-002 · C31. Observation was allowed to interfere with the system being observedEntif postmortem / incident recordThe act of observing the trainer became capable of disturbing it.
The surviving evidence could not reconstruct the exact causal chain inside that inaccessible runtime moment.
The operational lesson remained obvious: the observation path was invasive enough to enter the same risk surface as the process being observed.
That changes debugging behavior. The human starts thinking about whether it is safe to inspect the system while it is doing the thing that needs inspection.
And the pattern extends beyond debuggers.
Evaluators determine what gets preserved.
Audits create governance records.
Custody checks decide which artifacts remain live.
Baselines determine which debt remains visible.
Once an observer changes future state, observation has become part of the application.
8.10 Isolation multiplied into lifecycle debt
Git worktrees gave multiple AI agents separate working directories so they could edit and test different branches without overwriting one another.
That isolation solved a real problem.
We kept creating worktrees because the problem kept being real.
Eventually the visible Code location contained 79 linked worktrees. Git registered 129 when hidden roots were included. [C36][B:M119]C36 · Isolation, quota stewardship and learning records lacked lifecycle closureFailure Atlas deep caseWorktree isolation solved concurrent-edit risk by spawning a second lifecycle problem: dozens of physical workspaces without ownership, expiry, storage budgets, or trustworthy retirement.B:M119 · C36. Isolation, quota stewardship and learning records lacked lifecycle closureBilqis Mistakes ledger recordWorktree isolation solved concurrent-edit risk by spawning a second lifecycle problem: dozens of physical workspaces without ownership, expiry, storage budgets, or trustworthy retirement.
Each workspace now represented potential value, potential stale work, potential uncommitted state, branch ancestry, local experiments, duplicated payloads, and another object somebody had to classify before deletion.
The safety mechanism worked.
Its output became an estate-management problem.
This pattern appears everywhere in the Atlas.
Checkpoints preserve work, then require indexing and remote verification.
Soups preserve advisory models, then require custody and retirement.
Baselines preserve migration velocity, then require allowance hygiene.
Worktrees preserve concurrency, then require ownership and expiry.
Diagnostics preserve observations, then require integration into policy.
Postmortems preserve lessons, then require provenance and correction semantics.
Every control creates state.
Every state creates lifecycle.
Every lifecycle creates another opportunity for controls to disagree.
8.11 We kept solving reliability by manufacturing more things that could disagree
The pattern becomes brutally simple once you zoom out.
The original application had a failure.
We added machinery to contain the failure.
The machinery introduced a new representation of truth.
That representation gained readers and writers.
The readers drifted.
We added validation.
Validation acquired baselines, exceptions, historical state, and policy.
Those acquired lifecycle rules.
The lifecycle rules disagreed.
We added audits so the disagreement would be remembered.
The audits acquired their own identity and provenance problems.
At some point, the reliability system became a distributed system wrapped around the original application.
Distributed systems have a hobby.
They disagree.
The Failure Atlas documents us repeatedly paying for that hobby.