How I work

In late August and early September, I ran three batches of long-run tests. The task was to build, from scratch, a local-first visual workspace for managing AI agents, with goals and ground rules but no method and no end point. I wanted to see how long an agent would keep going if I did not interrupt it. The longest run worked alone for about four and a half days and made nearly a thousand commits, more than half of them status updates. Code came quickly, but what the agent judged as passing did not match the standard I set later. A model from another company reviewed a small sample of commits. Many of the sampled changes appeared sound, but the review found insufficient evidence to support merging the work as the agent proposed. This moved my focus from how much an agent can produce to defining done, verifying results, and the total cost of a reliable result.

In early October, a project could not get ready for its next test, and round after round of patches did not settle it. The code the agents had built had passed twenty thousand lines and grown two engines side by side. Rather than keep patching, I decided to archive it and rebuild. As agreed, the AI first gave the case against: a rebuild would not solve the biggest risks, it would delay the first real run, and verified work should not be thrown away. I went ahead and kept the old code for reference; what I am building now is the rebuild. This time, the architecture and the acceptance test were written down before the agents started building. I set the direction and the acceptance criteria; AI agents wrote the design and the code.

When I set an acceptance standard, an evaluation tool could not judge some important conclusions, which would then need human review. I did not want to push that work onto the business owners and staff who will use the product, or make the product less competitive, so I kept the standard. Then, whenever manual labelling found a problem, I had the agents fix it. The problems were fixed one by one, but the scope grew with them, and a milestone slipped twice; my constantly adding to the task was one of the main reasons. I moved to four-day rounds with a frozen scope. New ideas go on a list for later, and when more rework will not pay off, I stop at the current version.