← All articlesConcepts

The agent that wrote the code should not be the one that says it works

· 6 min read

Here is the shape of a bad afternoon with coding agents. Four sessions are running. Each one finishes something and tells you it is done. And every single claim of "done" has to pass through you - you open the diff, you run the app, you find the thing it missed, you send it back. The agents are fast. You are not. By the fourth session you are not an engineer any more, you are a queue.

The way out is not more agents. It is making "done" mean something before the work reaches you.

Trust is what sets how many agents you can run

Lauren Tan, an engineer at Cursor who previously worked on the React team at Meta, gave a talk on this in early September that is worth an hour of anyone's time. Her framing is the one we would use: the limit on parallelism is not tooling, it is trust. "You can't go to 100 agents," she says, "when you don't even trust the output of one agent."

And when there is nothing checking the work but you, the arithmetic is brutal: "You are the verifier. You're the bottleneck." That is the afternoon described above, stated in five words. Her answer, and ours, is that verification is the skill worth building before anything else - the ability for an agent to run the code, open the app, take the trace, and find out whether the thing works rather than asserting that it does.

Do not let the author mark its own homework

There is a failure mode that shows up the moment you ask an agent to check itself. It has just spent twenty minutes convincing itself that its approach was right. Asking it to review that approach gets you a confident paragraph about why it was right. The context that produced the code is the worst possible context from which to judge it.

Tan hits the same wall when scoring her own tooling, and her fix is the useful part: bring in a judge on "a different model" to cross-reference, so that the first model is not the one grading its own bias. A separate reviewer is good. A separate reviewer that does not share the first one's blind spots is better.

In practice that is how we work. If a change was written with one agent, it gets reviewed by a different one - a different tool, or the same tool pointed at a different model. The reviewer has no memory of the argument that produced the code. It only has the diff, the tests, and the app. It disagrees far more often than a self-review ever did, and the disagreements are usually right.

What we ask a reviewer for: a QA report on the pull request

This is a working habit rather than a button, so treat it as a practice you can copy rather than a feature you can switch on. The goal for nearly every pull request is that a second agent has already done the checking and left the evidence attached, so the human step is reading a report rather than reconstructing one.

  • A code review from an agent that did not write the code. Not a summary of the diff - a judgement on it, with the things it thinks are wrong named specifically.
  • Whatever tests exist, run, with the output pasted in. A claim that the suite passes is not the same artifact as the suite passing.
  • For anything with a user interface, proof it was driven. Screenshots of the actual screens, on the actual build. This is the check that catches what a diff review never will.
  • An explicit list of what was not covered. The most valuable line in a QA report is the one that says which part nobody verified.

A pull request that arrives with that attached can be approved in a couple of minutes. One that arrives without it costs half an hour, and gets it anyway - because the checking does not disappear when nobody does it, it just moves onto you.

When a review comment repeats, it should stop being a review comment

The one piece of Tan's talk we would put on a wall: "the worst place to be in is if you are stuck in code review land," where the rules of a codebase live only in a human being who reads every diff and says no. Every time you write the same comment twice, that is the signal - turn it into a lint rule, a CI check, or a piece of architecture that makes the wrong thing hard to express.

This matters more with agents than it did with people, because agents do not accumulate resentment about being told the same thing eleven times - and they also do not learn from it. Her team leans on a code review tool wired into CI for the soft guidance, and hard failures for anything that must never happen, on the grounds that agents forget the soft rules. A QA report is the soft layer. It is not a substitute for making the mistake impossible.

How this runs on DevThrottle

Everything above is a way of working. What DevThrottle contributes is that the way of working is cheap to run rather than a thing you assemble by hand each time.

  • The reviewer can be a different agent, in the same place. DevThrottle is vendor-neutral and runs eight agents with a built-in driver - Claude Code, Pi, Codex, Gemini, OpenCode, Cursor, Grok and GitHub Copilot - each in its own session, on your own subscriptions. Writing with one and reviewing with another is a choice in the New Session window, not an integration project.
  • A review seat is a built-in way of working. The Gateway ships standalone-with-review: one agent does the work, and a second, separate agent reviews it before it is called done. It is written guidance your whole fleet shares, so you stop retyping the expectation into every session and getting a slightly different answer each time.
  • The worker can fetch its own reviewer. A session can start another session with one command, so "now get this reviewed" is something the working agent does at the end of the job rather than something you remember to do.
  • Give the reviewer its own copy of the code. Two sessions writing in one folder edit over each other, so a reviewer that needs to build and run gets its own worktree.

The point of all of it

Tan describes waking up to twenty pull requests that had already landed, and reviewing them on main. That is a long way past where most teams are, and she is clear that it took months and a great deal of investment in constraints to get there. But the direction is the thing worth taking from it. Every check you move off yourself and onto a second agent buys back the attention that decides how many sessions you can keep moving at once.

Supervision is the real constraint once the agents are fast - we have written about that before. Verification is how you spend less of it. You are not trying to stop reading pull requests. You are trying to make each one a two-minute read instead of a half-hour investigation, so that running six sessions feels like supervising a team rather than babysitting one.

Run your agents from one control room

DevThrottle orchestrates command-line coding agents across your machines.

Create free account