Testing decision-challenge on six real projects

• It never touched my project. No message asked for an edit, but 21 of 126 runs with the short prompt and 43 with the plain request edited or deleted files already in the project, many of them after offering to make the change. With the command: 0.
• In real use it got more decisions right: 70%, against 60% for the short prompt and 51% for the plain request. In real use, a plain request stops at its first answer, while the command stops to ask for your priorities. In Claude Code alone, the short prompt kept up (45 of 54 each); the lead comes from the Copilot models.
• When the other two also got my priorities and a second turn, both jumped. The plain request and the short prompt each gained about 12 points, and the short prompt ran neck and neck with the command, scoring higher on some kinds of question (the prompt is below). The command takes two turns by design: it stops to ask for your priorities before it decides, where the other two answer in one. In real use, they stop at that first answer.
• It costs less than the short prompt given the same second turn, with fewer tool calls: $0.43 a run³ against $0.54 in Claude Code, and 5.8 Copilot credits against 8.8. In Claude Code, the short prompt's first answer on its own cost about half as much, and got as many decisions right.
• Where it slips: it's cautious: sometimes it pushes back on a plan that's actually fine, and sometimes it asks to measure something the project has already measured. Both are on my list for the next version.

¹ Real use. The short prompt and the plain request are marked on their first answer, where a real conversation usually ends; the command is marked on its answer after the priorities it asked for.

² Same information. Both control groups, the short prompt and the plain request, were given a second turn: my priorities, each marked high, medium or low. Only the command asks for priorities on its own (and offers a draft), so this way favors the other two: in real use, you'd have to plan that second message yourself and send your priorities without being asked.

³ One run. One decision, start to finish: both messages, one model. The dollar figure is Claude Code's own estimate at API prices, averaged over three models: for the command, about 4 cents a run with Haiku, 37 with Sonnet and 87 with Opus. On a Claude subscription, which is how I ran the test, you aren't billed per run; it counts toward your plan's usage limits. Copilot credits come from your plan's monthly allowance: for the command, about 1 to 2 a run on most models, and 18 on Gemini 3.8 Flash.

Keep in mind: six projects, one run per model and question, so treat a gap of a few points as noise. The decisions come from my own projects, and I wrote the questions.

Each project's page has its three questions word for word, the lines in the repo that decide each answer, and every model's mark, with some of the replies.

• tokenizer-lab: a lab measuring how vocabulary size changes small language models.
• sibling-site-drift-lab: a lab testing whether a checking routine stops AI agents leaving related files out of date.
• espalier-harness: my Claude Code harness. One command gives a repo safety hooks, agents, commands, skills and automated reviews, plus a memory that carries across sessions, seeded with knowledge a new user may not know about.
• role-aware-dashboard-toolkit: my toolkit for dashboards where what you see depends on your role, with a real auth layer: verified sign-in through your own provider, tied to roles the server grants.
• A social media orchestration engine: my harness that automates AI social media accounts and drafts posts for my personal, manually curated pages.
• specdec-sidechannels: my research on whether untapped signals from the device itself, like which parts of the model each word touched in memory, can speed up speculative decoding (a small model drafts the next words, a big model checks them).