Research with coding agents

How I run long investigations with Claude Code

I build most of the software on this site with coding agents (Claude Code, and some Codex) as day-to-day collaborators. About 190 of the roughly 490 commits in Rutter, and over 100 of the 142 in the video backlog transcode, were co-written with Claude. Writing the code turned out to be easy; keeping a long, branching investigation honest across hundreds of sessions did not.

For the electrode tracer, I set up a file-based lab notebook that the agent maintains and I review:

  • Every experiment is pre-registered with its question, prediction and method before it runs. Afterwards the entry records the result, what the result does not show, and the commit and command that produced it. The notebook holds over 200 entries.
  • Entries are append-only. A correction is a new entry citing the old one, and a small tool rejects edits, so the record can’t be quietly rewritten to fit a later story.
  • Resume, status and findings files, and a list of what has been tried, let a fresh session pick up the work in minutes without repeating dead ends.
  • The agent stops at decision points. Scope changes, GPU or cloud spend, and calls about what counts as a result come to me, and the notebook records who decided what.

With this structure, an agent has carried a months-long ML project, including cross-validated training on cloud GPUs, and still reports plainly when an idea fails.