The campaign

Thirteen modules audited over four days, plus two pre-campaign modules from a week earlier. Every module got the same treatment: four parallel CC subagents (Security Adversary, Correctness Pedant, Architecture Critic, Test Coverage Hunter), then web LLM rounds with Claude and ChatGPT until convergence.

Sorted by bug count. The largest modules produced the most bugs, but not proportionally — patch.py at 320 lines produced 32 bugs across 16 rounds. dedup.py, a dormant module with zero production catches, needed 8 rounds and produced 11 bugs.

The model that wouldn't stop finding bugs

The campaign ran Claude and ChatGPT in parallel every round. The pattern was consistent: Claude converged early, ChatGPT kept finding real bugs.

In github.py, ChatGPT found the non-string field rendering crash (Round 4), the line-zero API bug (Round 4), the corroborated_count type mismatch (Round 5), and the argv-size fragility in comment posting (Round 6) — all while Claude declared the module “production-ready.”

In dedup.py, ChatGPT peeled back seven consecutive layers of the same bug class: type guards on untrusted data. Same-reviewer comparisons, unsanitized prompt injection, malformed container shapes, synthetic ID collisions, invalid ID types, alias validation, and canonicalization gaps. Each round Claude PASSed. Each round ChatGPT found the next layer.

This is the thesis of Sweep playing out on Sweep itself. Ensemble diversity catches what same-model reviewers miss. Not because one model is smarter — because they have genuinely different blind spots. That's Mode 3.

The bug class that wouldn't die

One bug class appeared in every module: non-string values from LLM output reaching code that assumes strings. A finding with {title: 42} crashes .replace(). A classification with {classification: ["ADOPT"]} passes key-presence checks but fails string comparison. An ID of None maps to an empty-string dict key, silently overwriting prior entries.

The fix is always the same: validate type at the trust boundary, not just presence. Pydantic TypeAdapter for structured fields. A simple _safe_text() coercion helper for rendered fields. The pattern was so consistent that we codified it as gotcha #71.

The lesson: LLM output is not just “might be wrong.” It's “might be the wrong type entirely.” Every .get() that assumes the value is a string is a crash waiting for a model having a bad day.

The modules that surprised us

patch.py — 16 rounds, 32 bugs

A 320-line module that generates patches from adopted findings. It took more rounds than the 2,277-line triage module. Fence validation, identity checks, whitespace preservation, all-skipped accounting — the surface area was deceptive. Lines of code is not a predictor of audit complexity.

github.py — almost skipped

We nearly moved on to the next module after Round 3. The commit said “shipped v2.20, 3 rounds, 17 bugs.” Then we fed Round 4 to web LLMs. ChatGPT found 2 more P2s. Then 1 more. Then 1 more. Final count: 7 rounds, 21 bugs. The module that was “done” needed four more rounds.

dedup.py — 299 lines, dormant, 8 rounds

Zero production catches. The module exists as cheap insurance in case the lead model's dedup logic ever fails. It produced more audit rounds than types.py, eval/harness.py, sarif.py, chunk.py, and batch.py combined. Dormancy doesn't mean correctness.

The final numbers

528 regression tests means every fix has a test that prevents re-introduction. Seventy-two gotchas means every non-obvious design constraint is documented. The test suite went from 1,121 to 1,553. The source-to-test ratio is now roughly 1:1.

Before and after

Same artifact. Same four reviewers. Same lead model. The only difference: 278 bug fixes. We ran the full pipeline on the same codebase review that produced the v20 eval baseline, then compared.

Adopt rate collapsed from 52% to 29%. Not because the pipeline finds fewer bugs — because the bugs are already fixed. Twenty-eight findings were dismissed as ALREADY_ADDRESSED in Pass 2 alone. The verification gate caught 3 overpromoted P1s and auto-downgraded them to P2.

Reviewer signal shift

DeepSeek and GPT-Nano dropped from 73–75% signal to 27%. Not because they got worse — because they keep finding the same issues, and now the lead correctly dismisses them as already fixed. Grok's signal increased from 42% to 62%: it finds fewer but more novel issues that survive the stricter triage. The lead's own signal rose from 62% to 69%.

Severity calibration

P0 share dropped from 19% to 11%. P1 share rose from 11% to 25% — but with 100% precision and 3 auto-downgrades. The verification gate is doing its job: every P1 that reaches the changelog has code-path evidence. The confusion matrix is a perfect diagonal. No severity was ever misclassified.

The pipeline reviewing the pipeline

The methodology was designed to produce Mode 3 behavior: genuine independent discovery, not anchored verification. The campaign validated that this works — not just on spec documents and student code, but on the review pipeline's own source.

ChatGPT found bugs that Claude missed, round after round, in the same codebase. Not because one is better. Because they're different. That's the entire point. One model reviewing its own work converges to its own blind spots. Multiple models converge to the actual bugs.

278 bugs in code that was already working, tested, and production-deployed. Every one of them found by a model that had never seen the code before that round.

Independence is structural, not instructional.