How to know if an AI agent did the right thing
Traces, logs, and passing tests only show the steps an agent took, not whether the final outcome is actually correct. A run can finish cleanly while missing the intended goal or suddenly generating unexpected code. Because finished and succeeded are not the same thing, developers are left needing another agent just to check the numbers.
Core Friction Pattern
Analysis of 4 peer accounts across online developer and community forums reveals that “How to know if an AI agent did the right thing” repeatedly surfaces due to edge-case mismatches between standard defaults and real-world environments. Standard troubleshooting frequently falls short, compelling practitioners to develop custom workarounds.
+6 alternative community-tested workarounds cataloged below.
What the numbers show
Counted from the threads quoted below. How we count
- The most recent one is from November 2026.
- It is not one crowd: 2 people on Hacker News and 2 on Reddit ran into the same thing.
What people tried
Every workaround mentioned in the threads below. We haven’t tested any of them — and nobody here is claiming they worked.
Found another way around this?
Share your steps and trade-offs so fellow engineers don't have to reinvent the wheel. No account required.
How to know if an AI agent did the right thing
- 1Using a second, independent AI agent to recompute and verify the results.Did this help you?
- 2Experimenting with extracting intents from the agent and correlating among them.Did this help you?
- 3Applying traditional software unit tests and manual visual inspections to the final output.Did this help you?
- 4Decode the Base64 string into a file without executing it using `base64 -d` and inspect the contents with `less` before running.Did this help you?
- 5Building custom tools that read through agent conversations to generate detailed assessments.Did this help you?
- 6Sampling resolved, handed-off, and abandoned threads separately and manually checking claims against source records.Did this help you?
- 7Joining agent-resolved threads to later human tickets on the same contact to identify false resolutions.Did this help you?
In their words
Unedited, grouped by where they were said, most upvoted first within each place, each linked to the thread it came from.
“Traces and logs tell me what the agent did: every tool call, retry, token, and step. That's useful when something breaks. What I'm less sure about is whether that tells me the agent actually did the right thing.”source ↗
“A run can look clean and still miss what I wanted. Or it can "finish" without the outcome really being done. So finished and succeeded aren't the same thing.”source ↗
“All the first agents tests had passed. The traces tell me what it did, but I still need another agent to check the numbers.”source ↗
“We find that the built-in agent analytics quickly run out of depth when clients want to better understand how the agent is performing, the quality of its conversations, gaps, improvement needs, etc.”source ↗
“I asked Grok/Cursor for several fixes on my WHM/Cpanel via terminal command and one of the outputs was sudden base64 code. Scary AF?”source ↗
Where this came up
People with this problem also raised
Related community dilemmas clustered by discussion consensus
How to verify vendor AI claims
Vendors frequently use the buzzword "AI-ready" to disguise basic data deduplication scripts or make unverified claims that their built-in tools can replace dedicated services like Claude and ChatGPT. Because nobody explains these features in verifiable terms, buyers cannot determine if the software actually provides new value or can perform the same complex, time-saving tasks.
Why do I still have to check the UI after using AI?
AI tools and automated scripts frequently make mistakes, forcing people to manually inspect the main application interface to verify that tasks actually executed correctly. This defeats the time-saving purpose of automation by requiring constant babysitting and error correction.
How to audit employee AI tools and browser extensions
Auditing AI tools requires device-level visibility, because you cannot audit what you do not know exists when users install random browser extensions. Relying on a firewall to block access is a soft approach that simply pushes employees to use personal accounts and phones. To gain full clarity and control over AI usage, policies must be implemented directly on the device.
How do you test if AI-generated code actually works for your business?
AI code and formulas often run without crashing while producing completely wrong numbers or violating custom business logic. Because writing tests to catch these logical errors takes more time than generating the code, people find themselves unable to properly verify what's been built and end up harmed by hidden mistakes.
Mechanic changed battery CCA specification on test report
Mechanics sometimes manually input incorrect cold cranking amps (CCA) during a battery test, such as entering 800 CCA for a 640 CCA battery, which makes it impossible to trust the diagnostic results. Compounded by poor communication from the service department, vehicle owners are left unable to verify whether a component actually needs replacement or if the test data has been manipulated.
Where can I find real-world company AI policies and best practices?
There are no established industry standards or best practices around AI right now, leaving professionals to figure it out on their own. Trying to piece together a policy using AI or searching for other companies' examples falls short, creating a desperate need for outside data.