There’s a particular kind of disappointment that hits when an AI coding agent hands you a polished-looking pull request. The code looks clean. The comments are thoughtful. The structure even makes sense. Then you run the tests—or worse, you deploy it—and something fundamental is broken.
That moment is becoming rarer. Not because models suddenly got smarter overnight, but because the better agents stopped trusting themselves.
They started testing their own work.
What “Self-Testing” Actually Means
A self-testing agent doesn’t just generate code. It also tries to prove that the code works—before a human ever sees the change. Sometimes the same agent writes the implementation and then runs the existing test suite. Sometimes a separate model or specialized agent writes new tests designed to break the first agent’s output. The strongest systems do both.
This is different from the early wave of coding assistants that mainly completed functions or suggested snippets. Those tools were helpful, but they left verification almost entirely to the developer. The new generation treats verification as part of the job, not an afterthought.
The practical difference is huge. An agent that can write a feature, spin up the app, click through the critical path, run unit tests, and only then open a pull request is operating at a different level of reliability than one that simply dumps code into a branch and hopes for the best.
Why the Same Agent Testing Itself Isn’t Enough
There’s a well-known problem in software: people are terrible at proofreading their own writing. The same cognitive blind spots apply to AI. When one model generates both the code and the tests, it tends to test the assumptions it already made. It verifies its own mental model rather than challenging it.
This is why the more interesting systems deliberately separate the roles. One agent (or model) writes the implementation. A different one writes the tests that try to break it. First Mate Technologies has described exactly this pattern in public: one agent produces the code, a different model produces the adversarial tests. In one reported case they ran 554 test cases and surfaced 38 defects before the code was considered ready.
The separation matters. Different models have different failure modes. An adversarial testing agent is rewarded for finding problems, not for making the original agent look good. That shift in incentives produces more useful tests.
How the Better Agents Actually Work
Most production systems now follow some version of a closed loop:
- Understand the task and the existing codebase.
- Write the minimal change that should satisfy the requirements.
- Run the existing tests (and often generate new ones).
- Observe real execution results—failures, stack traces, screenshots, logs.
- Fix what broke and repeat until the evidence is clean.
- Only then open a pull request or mark the work complete.
Some agents go further. Devin can enter an explicit “test mode,” plan the verification steps, launch the application, and interact with it the way a human tester would. Cursor’s cloud agents and OpenAI’s Codex run inside virtual machines where they can build and execute the code. Greptile’s TREX feature runs each change in a disposable sandbox and returns traces and screenshots. OpenHands maintains a multi-layer verification stack that includes a critic model scoring work while the agent is still running.
GitHub’s Copilot coding agent has built a self-review loop into its workflow: it generates the change, runs its own review and security checks, iterates on the findings, and only then requests human attention.
The common thread is that verification is no longer optional. The agent is not allowed to declare victory until it has evidence.
Real Projects Already Using This Approach
This isn’t theoretical. Several systems are already shipping with self-testing capabilities:
- Devin treats testing as a first-class mode. It plans tests, runs the application, and returns annotated evidence.
- Cursor and Codex give agents real execution environments so they can build and test their own changes.
- OpenHands uses both lightweight critics and dedicated review/QA agents.
- First Mate’s internal workflow separates code generation from adversarial test generation and requires a full RED → GREEN → REVIEW → VERIFY cycle.
- Research systems such as SICA (self-improving coding agent), ReVeal, and Code-A1 explore even stronger forms of self-verification and adversarial co-evolution between code models and test models.
On the QA side, teams have built multi-agent “councils” that analyze features, write Playwright tests, debug failures, and expand coverage with minimal human intervention. One reported case grew test coverage from roughly 380 to over 700 tests while cutting flaky tests dramatically.
The Practical Benefits
When self-testing works, three things improve at once.
First, the quality of the first human-visible draft rises. Reviewers spend less time on obvious breakage and more time on architecture, product judgment, and edge cases the agent still misses.
Second, iteration speed increases. An agent that can fail, observe, and fix inside its own loop doesn’t need a human to notice every regression. The feedback cycle shrinks from hours to minutes.
Third, confidence in AI-generated code becomes more rational. Instead of hoping the model “got it right,” teams can demand executable evidence. Green tests, passing end-to-end flows, and clean security scans become the minimum bar for opening a pull request.
The strongest teams treat the agent the way they would treat a junior engineer who is fast but still needs guardrails: give it tools, force it to verify its work, and keep a senior person in the final approval seat.
The Remaining Gaps
Self-testing is not magic. Several limitations remain clear.
Same-model self-testing is still prone to collusion. The agent can write tests that are easy for its own code to pass. Separating the testing model helps, but it is not a complete solution.
Unit tests alone are not enough. Many of the most expensive failures live in integration points, timing, authentication flows, and real user journeys. Agents that only run unit tests can still ship broken features. Runtime verification against a realistic environment is harder and more expensive, but it catches a different class of bugs.
Coverage of edge cases and security properties is still uneven. Agents are better at happy paths than at adversarial inputs, race conditions, or subtle authorization mistakes. Human review and specialized security scanning remain necessary.
Finally, the cost of thorough verification is real. Running full test suites, spinning up environments, and iterating multiple times burns tokens and compute. Teams have to decide how much verification is worth the expense for a given change.
Where This Is Heading
The direction of travel is obvious. Coding agents that cannot demonstrate their work will look increasingly incomplete. The competitive bar is moving from “can generate plausible code” to “can generate code that survives its own tests and a realistic runtime check.”
We should expect more specialization: dedicated testing agents, better sandbox environments, stronger critics that score intermediate progress, and tighter integration between generation and verification. Multi-agent systems that deliberately set writers against testers will become more common precisely because single-agent self-checking has known failure modes.
The human role is also shifting. Instead of writing every test and catching every regression, experienced engineers increasingly design the verification strategy, set the quality gates, and review the evidence the agents produce. The bottleneck is moving from “can we generate the code” to “can we understand and trust what was generated.”
Closing Thought
Self-testing agents will not eliminate the need for careful engineering. They will, however, raise the floor. The days of accepting large AI-generated diffs on faith are ending. The better agents already refuse to ship until they have run the tests, observed the results, and fixed what they broke.
That change is quieter than the flashy demos, but it is the one that actually makes the technology usable at scale. Code that looks right is easy to generate. Code that survives contact with reality is still hard. The agents that can do both are the ones worth paying attention to.