We run a handful of AI agents across client accounts, and most of them do unglamorous work.
One sets up DNS records, another provisions mailboxes, and a third reads inbound replies so a human only sees the ones that matter.
That third one carries the most judgment. It reads what a prospect actually wrote and decides whether it counts as interest, a no, an out of office, or a referral.
A few weeks in, it started getting easy replies wrong, and blunt rejections were landing in the interested pile.
The cause turned out to be something we had done to it ourselves, one small fix at a time.
The agent got worse every time we improved it
The first version handled common replies well and struggled with unusual ones, which is roughly what you would expect.
So we did what everyone does, and added a line to the prompt for each miss we found.
They read like this: if the reply mentions a competitor, treat it as X unless it also mentions pricing.
Every addition fixed the example in front of us. After a dozen of them, the agent began misreading replies that matched none of those conditions.
The rules had started interacting, and the agent was no longer reading the email so much as working out which instruction outranked the others.
Why extra rules make judgment worse
A conditional rule is a narrow claim about one situation, and stacking enough of them shifts the model's attention.
It spends that attention reconciling our instructions instead of interpreting the message in front of it.
That is harmless when there is one correct answer, and genuinely bad when the task is interpretation, because real replies rarely match a condition cleanly.
We rewrote the prompt from eight conditionals down to two plain principles and three real examples.
Accuracy on the awkward replies went up rather than down.
What we kept
The shorter prompt holds three things, and nothing else survived the cut.
- A clear description of each category, written the way you would explain it to someone joining the team
- Two or three real examples per category, taken from actual client replies
- One instruction for uncertainty: flag it rather than guess
The edge cases those deleted rules were written for still turn up, and the agent handles most of them anyway.
It is reasoning from a principle now, rather than matching against a list of exceptions.
Where explicit rules still belong
This is not an argument against structure everywhere.
The agent that configures DNS records does not benefit from judgment, because an SPF record is either correct or it is not.
That work should stay as rigid and explicit as we can make it.
The distinction is the shape of the task rather than the technology: interpretation wants principles, configuration wants steps.
Our automation engineering work sits on both sides of that line, which is why we keep the two kinds of agent separate.
How we decide now
Before anything new goes into a judgment-based prompt, we ask whether it is a principle or a patch.
A principle is something a competent new colleague could apply to a case they have never seen.
A patch fixes one observed failure and quietly risks breaking an unrelated one.
If it reads as a patch, it goes into the examples section as a labeled instance, where the agent can reason from it rather than obey it.
We also watch the overall count, because past five or six rules in one prompt the agent starts weighing instructions against each other again.
When something breaks now, the first question is which rule to remove rather than which rule to add.
The Questions We Get Asked
Does this mean I should strip the rules out of my agent prompts?
Only the judgment-based ones. Tasks with a single correct answer, like DNS configuration or data formatting, still need explicit steps.
The pattern applies to classification, triage, and anything that involves interpreting what a person meant.
How do I know whether my prompt has too many rules?
If you cannot explain the agent's current behavior without tracing several conditionals interacting, it has too many.
A useful test is whether someone new could follow the instructions on a case they have never seen without getting stuck.
What should I do instead when the agent gets something wrong?
Add the mistake as a labeled example rather than a new conditional.
Examples let the agent reason by analogy, which generalizes better than a growing stack of exceptions.