Insights / Architecture
Where AI earns its place, and where a rule is better
Most workflows described as AI problems are deterministic problems with one ambiguous step inside them. Putting a model where a rule belongs costs money, latency, and the ability to explain what happened.
There is a reliable way to make an automation expensive, slow, and impossible to debug, and it is to use a language model for the parts of it that were never ambiguous. It happens constantly, because a workflow gets classified as an AI problem as a whole when in fact one step inside it is ambiguous and the rest is arithmetic, lookups, and conditionals.
The distinction is worth making precisely, because it determines cost per run, latency, failure modes, and whether anyone can explain to an auditor why the system did what it did.
The test
For any single step, ask: could a competent person write down the rule that decides this, such that two people following it would reach the same answer?
If yes, it is deterministic, and it should be code. Code is faster by orders of magnitude, effectively free per execution, produces the same output for the same input every time, and fails in ways a stack trace explains. No model competes with that on a decidable question.
If no, because the input is unstructured language, or the categories have fuzzy edges, or the right answer depends on context that resists enumeration, that step is a genuine candidate for a model. Those steps are real and they are usually a small fraction of the workflow.
The useful architecture is deterministic by default with a model at the specific points that need judgment. Not a model orchestrating everything, and not an attempt to write rules for genuinely ambiguous language. The boundary between the two is where the design work actually lives.
What this looks like in practice
Consider a common shape: inbound email arrives, something has to be extracted from it, a decision follows, and a record is updated somewhere.
Reading the email is a model problem. The text is unstructured, written by people, and phrased differently every time. Extracting an intent, an amount, or a reference from prose is exactly what these systems are good at, and no rule set covers it.
Validating what was extracted is not. Whether a reference number matches an existing record, whether an amount falls within an authorised range, whether a date is in the future, whether the sender is on the account: these are lookups and comparisons. Asking a model to do them is slower, costs money per check, and introduces the possibility of a wrong answer where none previously existed.
Deciding what happens next is usually not, either. If the policy is that requests above a threshold go to a human and requests below it proceed, that is a conditional. It should be visible in code where somebody can read it, change it, and point to it when asked why a particular request was escalated.
Writing the record is not. That is an API call with typed fields and defined error handling.
So one step of four uses a model. The other three are ordinary software, and building them that way means the workflow runs in milliseconds rather than seconds, costs a fraction as much, and produces an execution trace that reads as a sequence of decisions rather than an inference.
The four costs of putting a model where a rule belongs
Non-determinism where it is not wanted. The same input can produce different output. For a summarisation task that is tolerable. For deciding whether a payment is authorised it is a defect, and it is a defect that appears intermittently, which is the hardest kind to catch in testing.
Cost that scales with volume. A conditional costs nothing to evaluate a million times. An inference does not. At low volume the difference is invisible, which is precisely why it goes unnoticed until volume arrives.
Latency that compounds. Each model call adds time. A workflow with six of them where one was needed is several seconds slower per run, and if it sits in front of a person waiting, that is the difference between a tool people use and one they route around.
Loss of explainability. This is the one that matters most in regulated work. A conditional can be shown to an auditor. An inference can be described but not reproduced with certainty. When a system has to justify its decisions, every step that could have been a rule and was made a model instead is a step that can no longer be defended precisely.
The other direction is also an error
The opposite mistake is rarer but more painful: trying to write rules for something genuinely ambiguous. It presents as a growing library of special cases. Someone writes a keyword matcher for inbound messages, then adds exceptions for the phrasings it missed, then exceptions to the exceptions, and eventually there are four hundred lines that nobody understands and which still misclassify a fifth of the input.
The signal is the ratio of special cases to rules. A handful of exceptions is normal engineering. A system where the exceptions substantially outnumber the general cases is a system trying to enumerate something that resists enumeration, and that is the point at which a model earns its place.
Both errors come from deciding the architecture at the level of the workflow rather than the level of the step. Take the process apart, apply the test to each step individually, and the boundary is usually obvious. What is left is a system where the model does the part that needs judgment and code does everything else, which is cheaper, faster, and defensible in the room where somebody asks how it decided.