Human in the loop for AI agents works when the loop is one page, not a chat window. Every outward action my agents propose files as a row on a single approvals desk and waits for my click. That desk, a dumb CLI underneath it and sending caps enforced in code are why Kata Jaam, an event-discovery product I run alone, needs about fifteen minutes of my attention a day (as of 17 Oct 2026).
What goes on the desk
If an action leaves the building, it is a row: a DM to an organizer, an email, a social post, the approval of a listing, a paid placement, a delete. No agent does any of these on its own. The agents read, judge, draft and file. I click.
Kata Jaam runs on a rota of Claude Code agents with written briefs: inboxes, event scouting, catalogue QA, social drafts, SEO, retention, organizer success. They all file to the same page. Early on each agent reported into its own file, its own channel, its own log, and I spent my fifteen minutes finding the work instead of judging it. Putting every loop on one desk fixed that.
Designing the AI agent approval queue
Rows sort into four groups, and the order is most of the design:
- Failed after approve: I said yes and something broke. This is first because it is the only group where the world is already wrong.
- Overdue: a reply or a task that has waited past its window.
- Decisions: drafts that need a yes, an edit or a no.
- Approved and needs a hand: things I said yes to that still need a human step.
There are keyboard shortcuts for approve, edit and skip, so clearing a row takes one keystroke. The badge counts only what needs me now. A run log sits beside the desk and shows what each loop did and what it cost, so when a row looks odd I can see the run that produced it.
Why an approved row runs at once
An approved row executes immediately. There are no "send later" conditions inside drafts. A condition I cannot see is an action I did not approve. If an agent wants something sent on Friday, it files the row on Friday.
The guardrails under the desk
Under the desk, every write to production goes through one thin CLI that calls the admin's existing API. The CLI has no AI in it. Every verb has a dry run, a likely duplicate is refused, and it logs what it did. The agent decides the fields and the CLI writes them. There is exactly one path to production and I can read it.
The hard limits live in code too. Daily sending caps are enforced by the system. A prompt is only advice to a model, so a limit that matters cannot live there. Each channel has its own daily cap.
Start with sending off, widen per loop
Every loop starts in phase one: it files drafts and sends nothing. I widen autonomy per loop, and only when the approval rate says the drafts are right.
Eight months in, outward messages still wait for a click, and that is fine. Approving a good draft costs a keystroke. Recalling a bad DM to an organizer costs more than that.
Why not full autonomy or chat-style confirmation
Full autonomy means the agent acts and you read the log afterwards. It looks efficient until the first wrong message reaches a real person. Then you are cleaning up in public, and the only fix available is to turn the agent off.
Chat-style confirmation means the agent stops and asks "shall I send this?" at every step, inside a conversation. It keeps a human in the loop, but it puts the human in the agent's loop, on the agent's schedule. You have to be present while it works, and the questions arrive one at a time with no ordering.
The desk sits between the two. The agents work when they work, and I decide in batches, sorted by urgency, a keystroke per row. How much a loop may do alone is a per-loop setting, and the approval rate decides it.
The same idea inside a product
Kai, the coach I built inside Livio, is the same discipline pointed at a user instead of a founder. It calls thirty-plus tools to read and change a member's real health data, and every change it proposes is confirmed by the member with one tap. Numbers are computed in code, never by the model. The Kai case study has the details.
What the desk does not solve
The desk catches what I can see. It does not catch a judgement error that looks fine. A polite, well-formatted draft built on a wrong reading of the situation will get my click, because it looks right in the row. The usual shape is a reply that answers the question the organizer asked when they meant a different one. The weekly retro loop reads the journal and proposes diffs to the briefs, which is how those errors become rules, but only after they happen.
It also makes me the bottleneck. Nothing outward moves until I clear the page. On a day I do not open it, the company waits. Overdue rows sort above everything except failures, so the next time I open the page the replies that have waited longest are near the top.
And it does not move the numbers that matter most. Week-four retention at Kata Jaam is under ten percent (as of 17 Oct 2026). The retention loop reads the cohorts every Thursday and drafts the weekend push plan, and I approve it.
If you want one
I build this kind of agent setup, approval queue first, for other businesses: AI operations.