Imagine spinning up an AI out-of-office reply for your boss and discovering it costs $10,000 a day. That’s what happened to Retool CEO David Hsu when a well-intentioned employee learned the hard way what unchecked AI automation can cost.
The mistake? Instead of the autoreply simply stating that Hsu was away, the automation repeatedly scanned Microsoft Teams channels asking, in effect: Do I need to reply to this? Do I need to reply to that?
The agent chewed through millions of AI tokens. The bill reached roughly $10,000 a day, according to Hsu’s recount to attendees of the Leadership Council Summit hosted by the Wall Street Journal last week.
The gap between what AI costs and what it delivers is becoming difficult to measure. In cybersecurity, the same visibility problem carries a second price: risk.
An agent that can quietly burn millions of tokens can also call tools, inherit permissions, spawn subagents and act on sensitive data. If an enterprise cannot tell what an agent was doing, what context it was using and whose authority it was acting under, it may have trouble explaining not just the bill – but the breach.
“That is a cost problem and an incident-response problem wearing the same clothes,” said Evgenii Arsentev, PhD, a digital-health CEO and researcher who has been studying the economics of agentic AI.
For CIOs and CISOs, the AI boom has created an awkward mandate: spend enough to keep up, but be prepared to prove the money is producing something useful.
That is particularly difficult in cybersecurity. The cheapest model may miss the vulnerability. The most capable model may burn expensive tokens on work a smaller model could handle. An autonomous agent may save analyst time while creating new security exposure of its own.
What Should a Security Outcome Cost?
A Security Point Break examination of AI cost and risk suggests the hard question is no longer simply what a million tokens cost. It is what a useful security outcome should cost and whether enterprises can tell the difference between necessary investment, expensive waste and savings that quietly increase risk.
The cheapest AI is not necessarily the most economical AI, especially when the thing being priced is security risk.
Arsentev puts the trade-off bluntly: “A 30 percent saving on inference against an unmeasured miss rate is not a trade, it is a guess with a spreadsheet attached.”
Put simply: Saving money on AI means little if nobody knows whether the cheaper system is also missing more threats.
OWASP now classifies uncontrolled AI consumption as a security risk. Forcepoint, citing OWASP modeling, shows how quickly the math can turn ugly. Because an agent may reprocess its accumulated history with every new exchange, the estimated cost of a single response can rise from a fraction of a cent at the start to about 50 cents by the 100th turn.
Forcepoint X-Labs highlighted another wrinkle in its report released Monday. Runaway spending does not necessarily require an attacker. A poorly configured automation or an agent that simply does not know when to stop can inflict its own “denial of wallet” – consuming paid AI resources until the bill becomes the damage.
To demonstrate the problem, Forcepoint built a simulated research agent with no call budget or recursion limit. Fed a poisoned data source, the agent cascaded into 500 tool calls and $10 in simulated costs before researchers stopped it. With call budgets, recursion limits and other guardrails in place, the same test stopped after one call at a cost of two cents.
While those controls can stop an agent from spending indefinitely, they cannot answer a harder question. Is the agent wasting money, or does it simply need more computing power to finish an important job?
For a CISO, cutting off an agent that is stuck in a loop saves money. Cutting off one that is still investigating an attack could do the opposite.
That forces enterprises to ask not simply how much AI should be allowed to spend, but how much intelligence a particular job actually requires.
Use the Right Model for the Right Job
One response is to stop sending every AI task to the newest, most capable and often most expensive model available, said Michael Leland, field CTO at enterprise-browser company Island.
Developers may need Anthropic, lawyers may work in specialized systems such as Harvey or LexisNexis, while other employees may be perfectly well served by general-purpose models, Leland said.
“Why would you burn down the most expensive tokens of the latest models” on work that does not require them?
Island already allows companies to steer different users toward approved models. Leland said the company is working toward what it calls “model fit steering”. This is where model use is based not simply on who is asking, but on the task itself.
“Based on the prompt, we say what’s the best model to answer this question at the right cost,” Leland said.
Island also collects telemetry on token use, cost, sessions and MCP calls, giving companies a way to see who is consuming what. This, he said, gives accounting the ability to charge AI spending back to individual departments.
But Leland’s point goes beyond accounting. Much of the same telemetry used to understand what an agent costs can also show security teams what it did.
Island can retain prompts and responses along with tool calls and tool results. Its MCP gateway can also broker an agent’s permissions. If an agent asks for read-write access to Salesforce, for example, policy can restrict it to read-only.
Agent auditing is important for a variety of reasons, he said, because agents do not always follow the path their users expect. Leland said he watched one coding agent get stuck on a task, write a Python script, launch a terminal and retrieve an npm package from a vulnerable source.
“If left to its own devices, it’s going to do things that you didn’t intend or expect,” Leland said.
That is where architecture starts to matter.
When Cheaper AI Costs More
Leland’s approach can lower the price of the intelligence being used. But in security, cheaper is only cheaper if it works.
That is where the math gets slippery. AI productivity gains show up quickly. Security failures often send the bill later.
“The speed and the ability to turn a profit always is the primary concern of business,” said Darren Meyer, security research advocate at Checkmarx. Security is harder to price because “everybody’s secure until the moment they’re not.”
The same tension exists inside security tools. AI can reason deeply about a vulnerability and unpredictably miss the next one. Traditional static analysis catches more, but often brings a blizzard of noise.
“Using AI to do security work has some problems,” Meyer said. “It tends to be very good at certain things, but it tends to not be very predictable, and it tends to miss a lot of things.”
Checkmarx combines traditional static analysis with AI-based analysis. Across seven production codebases, the company said its hybrid engine scored 0.64 on the F1 measure, versus a 0.20 average for competing approaches it tested, while cutting false positives 60%.
The larger lesson is that AI can make one line on the spreadsheet smaller while quietly moving the cost somewhere else, such as missed vulnerabilities, added risk or a human left cleaning up.
Meyer suspects some companies are betting those problems disappear first.
“There is definitely a hype aspect to AI,” he said, with companies making “very, very big bets” that today’s shortcomings will be solved before the bill comes due.
“I think we are going to find out which organizations have over invested in AI, which ones have under invested, and kind of who got it right.”
Gartner sees that reckoning coming. It expects agentic-AI investment to grow 31.8%, even as it predicts more than 40% of agentic AI projects will be canceled by the end of 2027 over cost, unclear value or inadequate risk controls.
Companies, in other words, are spending more while learning what not to spend it on.
The Model Isn’t the Bill
A cheaper model can reduce the price of a workload. It cannot rescue a workload designed to gorge on AI. Christian Schnedler, CEO and co-founder of Rilian, calls that an architecture problem.
“The shortcut that they’re taking is creating a huge dependency on the LLM itself and using the LLM as a crutch,” Schnedler said.
The problem is not just which model an agent calls. It is how much baggage the agent brings along every time it calls one.
An agent working through a long task can accumulate chat history, documents, tool results, instructions and earlier decisions. If all of that context gets shipped back to the LLM every time the agent needs to think, the meter keeps running on information the model may have already seen.
Rilian’s approach is to make the model carry less of that load: smaller context windows, less information sent with each call and more filtering and routine work handled outside the LLM by software and conventional tools.
The distinction matters. Switching from an expensive frontier model to a cheaper one can reduce what each token costs. But if the system is still feeding millions of unnecessary tokens through it, all you have done is negotiate a better price on the waste.
Arsentev puts it more elegantly. “Routing is a discount on a bill you should not be receiving,” he said.
That changes the economics of the conversation. The goal is not simply to buy cheaper tokens. It is to need fewer of them — and to know whether the ones you paid for accomplished anything.
For CIOs and CISOs, that suggests the useful unit of AI may not be the token at all. It may be the outcome: the vulnerability found, the alert investigated, the incident contained or the hour of human work actually eliminated.
The question becomes less What did the AI cost? And more What did we get for the money?
Price the Outcome, Not the Token
There may be no “right price” for AI. An inexpensive model doing pointless work will always be waste. An expensive agent that stops an attack may be a steal.
That suggests a better measure: price the outcome, not the meter.
Arsentev said enterprises should look at cost per completed task, including retries and human intervention. In cybersecurity, he would narrow that further to cost per confirmed finding, measured alongside the false-negative rate.
“People optimize the number they can see,” Arsentev said. Tokens are wonderfully visible, he said, when they arrive neatly itemized on the bill. Whether those tokens produced anything worth the price is another story.
McKinsey’s Kate Smaje put numbers around that problem at the Wall Street Journal’s Technology Council Summit. More than 80% of executives in recent firm research reported personal productivity gains from AI. About 37% reported measurable earnings improvement. Just 6% were seeing value significant enough to matter to investors.
Saving somebody 20 minutes is useful. It is not automatically 20 minutes a CFO can take to the bank. Retool’s Hsu offered a more practical rule for the AI era.
Companies need to “enable a lot,” Hsu said — give teams room to test many AI use cases, then shut down the ones that fail to prove their value and invest harder in the ones that do.
For CIOs and CISOs, the discipline may not be spending less on AI. It is knowing what the money bought.
Sometimes it stops an attack. Sometimes it buys a $10,000-a-day message saying the boss is out of the office.
