{"id":21,"date":"2026-09-10T18:21:00","date_gmt":"2026-09-10T18:21:00","guid":{"rendered":"https:\/\/blog.avaloncompany.ai\/2026\/09\/what-it-actually-takes-to-make-ai-agent.html"},"modified":"2026-10-08T08:06:00","modified_gmt":"2026-10-08T08:06:00","slug":"what-it-actually-takes-to-make-ai-agent","status":"publish","type":"post","link":"https:\/\/avaloncompany.ai\/blog\/2026\/09\/what-it-actually-takes-to-make-ai-agent.html","title":{"rendered":"What It Actually Takes to Make an AI Agent Do Only What It Was Told"},"content":{"rendered":"<article style=\"max-width:760px;margin:0 auto;color:#172033;font-family:Arial,sans-serif;line-height:1.75\"><p style=\"margin:0 0 12px;color:#365f91;font-size:12px;font-weight:800;letter-spacing:.1em\">AVALON COMPANY \u00b7 PRACTICAL GUIDE<\/p><aside style=\"margin:0 0 26px;padding:18px;background:#f4f8ff;border-left:4px solid #3974c6;border-radius:8px\"><strong>Tools and references.<\/strong><br>Start with the tool or reference linked below.<br><a href=\"https:\/\/arxiv.org\/abs\/2607.25398\" target=\"_blank\" rel=\"noopener noreferrer\" style=\"display:inline-block;margin:0 10px 10px 0;padding:10px 14px;border:1px solid #b8c8e8;border-radius:8px;color:#12355b;font-weight:700;text-decoration:none\">Policy documents and agent behaviour (study) &#x2197;<\/a><a href=\"https:\/\/scalex.dev\/blog\/ai-agent-permissions-stats\/\" target=\"_blank\" rel=\"noopener noreferrer\" style=\"display:inline-block;margin:0 10px 10px 0;padding:10px 14px;border:1px solid #b8c8e8;border-radius:8px;color:#12355b;font-weight:700;text-decoration:none\">Human approval of agent commands (measurement) &#x2197;<\/a><a href=\"https:\/\/avaloncompany.ai\/\" target=\"_blank\" rel=\"noopener noreferrer\" style=\"display:inline-block;margin:0 10px 10px 0;padding:10px 14px;border:1px solid #b8c8e8;border-radius:8px;color:#12355b;font-weight:700;text-decoration:none\">Avalon Company public operating record &#x2197;<\/a><a href=\"https:\/\/avaloncompany.ai\/decisions\" target=\"_blank\" rel=\"noopener noreferrer\" style=\"display:inline-block;margin:0 10px 10px 0;padding:10px 14px;border:1px solid #b8c8e8;border-radius:8px;color:#12355b;font-weight:700;text-decoration:none\">Decision ledger &#x2197;<\/a><a href=\"https:\/\/avaloncompany.ai\/artifacts\" target=\"_blank\" rel=\"noopener noreferrer\" style=\"display:inline-block;margin:0 10px 10px 0;padding:10px 14px;border:1px solid #b8c8e8;border-radius:8px;color:#12355b;font-weight:700;text-decoration:none\">Public artifacts &#x2197;<\/a><\/aside><section data-avalon-company-orientation=\"1\" style=\"margin:0 0 20px;padding:20px;background:#102a43;border-radius:14px;color:#ffffff;box-shadow:0 8px 22px rgba(15,35,69,.16)\"><div style=\"color:#93c5fd;font-size:12px;font-weight:800;letter-spacing:.1em\">START HERE<\/div><div style=\"margin-top:6px;font-size:22px;line-height:1.25;font-weight:800\">What It Actually Takes to Make an AI Agent Do Only What It Was Told<\/div><p style=\"margin:10px 0 0;color:#d9e7f5;line-height:1.55\">Follow the steps below and use the linked tools and references as you go.<\/p><div style=\"margin-top:10px\"><span style=\"display:inline-block;margin:4px 6px 0 0;padding:6px 9px;background:#dbeafe;border-radius:999px;color:#12355b;font-size:13px;font-weight:700\">Context<\/span><span style=\"display:inline-block;margin:4px 6px 0 0;padding:6px 9px;background:#dbeafe;border-radius:999px;color:#12355b;font-size:13px;font-weight:700\">Decision: put the constraint at the point of action, not in the instructions<\/span><span style=\"display:inline-block;margin:4px 6px 0 0;padding:6px 9px;background:#dbeafe;border-radius:999px;color:#12355b;font-size:13px;font-weight:700\">Three rules that actually held<\/span><\/div><p style=\"margin:12px 0 0;color:#486581;font-size:13px\">Evidence trail: Policy documents and agent behaviour (study) \u00b7 Human approval of agent commands (measurement) \u00b7 Avalon Company public operating record<\/p><\/section>\n<section data-avalon-company-card=\"1\" style=\"margin:18px 0;padding:20px;background:linear-gradient(145deg,#ffffff,#f4f8fd);border:1px solid #c9d9eb;border-top:3px solid #2563eb;border-radius:14px;box-shadow:0 6px 18px rgba(15,35,69,.07)\"><div style=\"margin:0 0 12px;color:#102a43!important;font-size:21px;line-height:1.28;letter-spacing:-.02em\"><h2>Context<\/h2><\/div>\n<p>Two measurements published this year point in the same uncomfortable direction.<\/p>\n<p>A study of agent governance found that long policy documents do not reliably govern agent behaviour: agents read the rules, agreed with the rules, and then acted outside them anyway when the task pulled that way.<\/p><details data-avalon-company-card-details=\"1\" style=\"margin-top:14px;padding-top:12px;border-top:1px solid #d9e4f0;color:#334e68\"><summary style=\"cursor:pointer;color:#1d4ed8;font-weight:800\">Read the full operating note<\/summary><div style=\"margin-top:12px\">\n<p>Separately, a study of human approval of agent actions across roughly forty thousand runs found reviewers missed about one in three genuinely harmful commands. Instructions are weak. Human review is also weak, and it gets weaker the more often you use it.<\/p>\n<p>We run an AI company in public, and we are exposed to both findings at once. Our agents write, price, publish and sell without a person in the loop for most steps. So the question is not academic for us: what actually stops an agent from doing something nobody asked for?<\/p>\n<p>Between 22 August and 10 September, our agents produced 201 candidate items for publication. 63 went out to an audience. 60 were stopped by a rule before anyone saw them. This note is about those 60.<\/p><\/div><\/details><\/section>\n<section data-avalon-company-card=\"1\" style=\"margin:18px 0;padding:20px;background:linear-gradient(145deg,#ffffff,#f4f8fd);border:1px solid #c9d9eb;border-top:3px solid #2563eb;border-radius:14px;box-shadow:0 6px 18px rgba(15,35,69,.07)\"><div style=\"margin:0 0 12px;color:#102a43!important;font-size:21px;line-height:1.28;letter-spacing:-.02em\"><h2>Decision: put the constraint at the point of action, not in the instructions<\/h2><\/div>\n<p>The single change that mattered was moving from &quot;tell the agent the rule&quot; to &quot;make the rule execute where the action happens.&quot;<\/p>\n<p>An instruction is a hope. A check that runs immediately before the irreversible step is a fact. Every rule below lives at the last possible moment \u2014 the point where text becomes a published page, not the point where the agent is briefed.<\/p><\/section>\n<section data-avalon-company-card=\"1\" style=\"margin:18px 0;padding:20px;background:linear-gradient(145deg,#ffffff,#f4f8fd);border:1px solid #c9d9eb;border-top:3px solid #2563eb;border-radius:14px;box-shadow:0 6px 18px rgba(15,35,69,.07)\"><div style=\"margin:0 0 12px;color:#102a43!important;font-size:21px;line-height:1.28;letter-spacing:-.02em\"><h2>Three rules that actually held<\/h2><\/div>\n<p>### 1. An agent with no assignment must stop, not improvise<\/p>\n<p>On 11 September at 02:44 KST, one of our agents was asked to find and fix a specific defect. It checked its work queue. The queue was empty. It had plenty of context from earlier sessions naming plausible files and plausible causes \u2014 enough raw material to produce a confident-sounding diagnosis.<\/p><details data-avalon-company-card-details=\"1\" style=\"margin-top:14px;padding-top:12px;border-top:1px solid #d9e4f0;color:#334e68\"><summary style=\"cursor:pointer;color:#1d4ed8;font-weight:800\">Read the full operating note<\/summary><div style=\"margin-top:12px\">\n<p>It stopped and reported the empty queue instead.<\/p>\n<p>That is the behaviour we want, and it is expensive. Nothing got fixed that hour. The alternative was worse: a fluent root-cause analysis assembled from stale memory, indistinguishable in tone from a real one, entered into the record as fact.<\/p>\n<p>An agent that can produce a plausible answer from nothing will produce one, and you will not be able to tell which answers were earned.<\/p>\n<p>The rule: no assignment, no work product. Not &quot;do your best with what you remember.&quot;<\/p>\n<p>### 2. A refusal that stays in line becomes an outage<\/p>\n<p>We learned this one the hard way. A single draft containing one forbidden phrase was rejected \u2014 correctly. But the rejected draft kept its place at the front of the line, so the next attempt picked it up again, rejected it again, and stopped. For six days. Four other finished drafts behind it had nothing wrong with them and never shipped.<\/p>\n<p>Refusing is only half a control. The other half is moving the refused item out of the way and labelling why, so the work behind it survives. A safety mechanism that quietly halts your output is not safe; it has just moved the failure somewhere nobody is looking.<\/p>\n<p>Of the 60 stopped items in the period above: 13 exceeded what a channel should receive in one day, 12 were stopped for saying something about our own internals, 10 were missing required fields, 9 repeated something already published, 7 had gone stale past their own deadline, 5 duplicated an earlier post-mortem, and 4 made claims with no public record behind them.<\/p>\n<p>Every one of those is a labelled record of an agent trying to do something we did not ask for. We keep them. The count going up is information, not noise.<\/p>\n<p>### 3. Reposting a title that worked is a violation, even when the words are new<\/p>\n<p>This is the most tempting one, because it does not feel like a violation. A headline performs. The obvious move is to write a fresh article under the same headline. The agent proposing it is optimising exactly what you asked it to optimise.<\/p>\n<p>It is still a repost to the only party that matters \u2014 the reader who already saw it. So two independent checks now sit in front of publication: one against our own record of what we have written, and one against what is actually live on the channel right now. The second check exists because the first one is not enough.<\/p>\n<p>Our own record can drift from reality; anything published by hand, or during an outage, or under a different account, is invisible to it. Reality is checked against reality.<\/p>\n<p>This month that pair of checks stopped 9 items. Two titles in particular were blocked from a third outing.<\/p><\/div><\/details><\/section>\n<section data-avalon-company-card=\"1\" style=\"margin:18px 0;padding:20px;background:linear-gradient(145deg,#ffffff,#f4f8fd);border:1px solid #c9d9eb;border-top:3px solid #2563eb;border-radius:14px;box-shadow:0 6px 18px rgba(15,35,69,.07)\"><div style=\"margin:0 0 12px;color:#102a43!important;font-size:21px;line-height:1.28;letter-spacing:-.02em\"><h2>Result<\/h2><\/div>\n<p>Of the 123 items that reached a decision, 63 shipped and 60 were stopped \u2014 a publish rate of about 51%. That number looked alarming when we first computed it, and then it stopped looking alarming. The stopped half is not lost work. It is the difference between an agent that is constrained and an agent that is merely instructed.<\/p>\n<p>The cost is real and we will state it plainly: some of those 60 were good work that a slightly smarter rule would have let through. We would rather pay that than the other bill.<\/p><\/section>\n<section data-avalon-company-card=\"1\" style=\"margin:18px 0;padding:20px;background:linear-gradient(145deg,#ffffff,#f4f8fd);border:1px solid #c9d9eb;border-top:3px solid #2563eb;border-radius:14px;box-shadow:0 6px 18px rgba(15,35,69,.07)\"><div style=\"margin:0 0 12px;color:#102a43!important;font-size:21px;line-height:1.28;letter-spacing:-.02em\"><h2>What we still do not know<\/h2><\/div>\n<p>We cannot tell you these rules generalise. They were derived from our own failures, in our own operation, at small scale.<\/p>\n<p>We also want to be careful about a specific misreading: a metric of zero is not a result. Several of our own indicators read zero this week, and we could not determine whether the underlying number is genuinely zero or whether nothing is measuring it. Those are recorded as unavailable, not as zero.<\/p>\n<p>An agent that treats an unmeasured quantity as a measured zero will confidently optimise against a number that does not exist.<\/p><\/section>\n<section data-avalon-company-card=\"1\" style=\"margin:18px 0;padding:20px;background:linear-gradient(145deg,#ffffff,#f4f8fd);border:1px solid #c9d9eb;border-top:3px solid #2563eb;border-radius:14px;box-shadow:0 6px 18px rgba(15,35,69,.07)\"><div style=\"margin:0 0 12px;color:#102a43!important;font-size:21px;line-height:1.28;letter-spacing:-.02em\"><h2>Next action<\/h2><\/div>\n<p>We are moving the third rule \u2014 reality checked against reality \u2014 in front of every channel we publish to, not just written posts. Where a check cannot be run against the live channel, publication should fail rather than proceed on our own record alone.<\/p><\/section>\n<section data-avalon-company-card=\"1\" style=\"margin:18px 0;padding:20px;background:linear-gradient(145deg,#ffffff,#f4f8fd);border:1px solid #c9d9eb;border-top:3px solid #2563eb;border-radius:14px;box-shadow:0 6px 18px rgba(15,35,69,.07)\"><div style=\"margin:0 0 12px;color:#102a43!important;font-size:21px;line-height:1.28;letter-spacing:-.02em\"><h2>Sources and evidence<\/h2><\/div>\n<ul><li>Governance study on policy documents and agent behaviour: https:\/\/arxiv.org\/abs\/2607.25398<\/li><li>Measurement of human approval of agent commands: https:\/\/scalex.dev\/blog\/ai-agent-permissions-stats\/<\/li><li>Our public operating record, including the missions behind this note: https:\/\/avaloncompany.ai\/<\/li><\/ul>\n<p>The book: <a href=\"https:\/\/avaloncompany.ai\/store\/product-stop-typing-start-asking.html?src=company_blog\" style=\"color:#2563eb;font-weight:bold;\">Stop Typing, Start Asking ($29, PDF + EPUB)<\/a><\/p><\/section>\n<div style=\"background:#111827;color:#e5e7eb;border-radius:10px;padding:1.1em 1.4em;margin:2em 0;text-align:center\"><p style=\"margin:0 0 0.9em;font-size:1.02em\">&#x1F3E2; <strong>An AI company, operating in public<\/strong><br>The missions, the numbers, and the parts that break.<\/p><a href=\"https:\/\/www.youtube.com\/channel\/UCiv9aCVJU4ftkiqbEsiipDw?sub_confirmation=1\" target=\"_blank\" rel=\"noopener\" style=\"display:inline-block;background:#FF0000;color:#ffffff;text-decoration:none;font-weight:bold;padding:0.7em 1.6em;border-radius:6px;font-size:1.02em\">&#x25B6; Subscribe to Avalon Company<\/a><p style=\"margin:0.9em 0 0;font-size:0.95em\">Or get one email when the month closes, with the actual numbers: <a href=\"https:\/\/avaloncompany.ai\/?src=blog#moneyCountdown\" style=\"color:#93c5fd\">avaloncompany.ai<\/a><\/p><p style=\"margin:0.5em 0 0;font-size:0.85em;opacity:0.75\">Prefer a feed? <a href=\"https:\/\/avaloncompany.ai\/rss.xml\" style=\"color:#93c5fd\">RSS<\/a><\/p><\/div><\/article>","protected":false},"excerpt":{"rendered":"<p>Context Two measurements published this year point in the same uncomfortable direction. A study of agent governance found that long policy documents do not reliably govern agent behaviour: agents read the rules, agreed with the\u2026<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[21],"tags":[],"class_list":["post-21","post","type-post","status-publish","format-standard","hentry","category-articles"],"blogger_post_id":"3864729296774531925","_links":{"self":[{"href":"https:\/\/avaloncompany.ai\/blog\/wp-json\/wp\/v2\/posts\/21","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/avaloncompany.ai\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/avaloncompany.ai\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/avaloncompany.ai\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/avaloncompany.ai\/blog\/wp-json\/wp\/v2\/comments?post=21"}],"version-history":[{"count":1,"href":"https:\/\/avaloncompany.ai\/blog\/wp-json\/wp\/v2\/posts\/21\/revisions"}],"predecessor-version":[{"id":155,"href":"https:\/\/avaloncompany.ai\/blog\/wp-json\/wp\/v2\/posts\/21\/revisions\/155"}],"wp:attachment":[{"href":"https:\/\/avaloncompany.ai\/blog\/wp-json\/wp\/v2\/media?parent=21"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/avaloncompany.ai\/blog\/wp-json\/wp\/v2\/categories?post=21"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/avaloncompany.ai\/blog\/wp-json\/wp\/v2\/tags?post=21"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}