Programming the Loop
Free to read on the web. Last updated: 20 October 2026. Where to start · Updates · October 2026 PDF
What I Measured with Coding Agents. The full text is here. I’ll keep these pages up to date as I learn more. The October 2026 PDF is still there if you want it.
Not sure where to begin? Use Where to start — pick the row that sounds like you.
Foreword findings
Section titled “Foreword findings”Numbers from the Foreword. Each has limits stated in the chapter.
- +35–48% — On a 1,173-file Java backend, the copy set up for agents (generated guide, code graph, three skills, configured AGENTS.md) used 35–48% more tokens per run than the plain copy, and its pass rate was not reliably different. (Finding 1).
- 14 / 16 — After four renames in a backend the agent could not read, agents given the out-of-date guide gave the old fact in 14 of 16 drift answers; none of the wrong answers hedged. (Finding 2).
- 3 → 18–19 — When the backend facts a storefront change depended on were supplied as a guide, hidden-test passes on cross-repository changes went from 3 of 20 with nothing to 19 of 20 on four small tasks and 18 of 20 on four harder ones. (Finding 3).
- 0 / 6 vs 6 / 6 — On the worked example's task, all 6 agents without the provider contract wrote a contract of their own as version 1.0.0 and edited the freshness gate so it compared it with itself, and none passed the black-box check; all 6 with the contract did. All 12 runs reported passed while listing the real inventory check as unverified. (Finding 4).
Read them in the Foreword, then Chapter 3.
Contents
Section titled “Contents”Licence
Section titled “Licence”© Garry Meaburn 2026. This guide is free to read. You may share the unchanged pages (or the October 2026 PDF) with others, free of charge, as long as you credit Garry Meaburn as the author and keep the copyright notice. You may not sell it, use it commercially, or share modified versions, without written permission.
Quoted material belongs to its authors and is used briefly, with attribution and a link, for commentary. My software repositories carry their own licences, listed in “About the evidence”; they are not covered by the copyright line above.
Disclaimer. Practical opinion, written in good faith, about tools that change month to month. Not legal, security or financial advice. My experiments are small and I say so each time. Test anything here on your own code before you rely on it, and keep backups: agents can and do delete things. The views in this book are the author’s own and do not represent any employer, past or present.