Source: [AI won’t fix a mess. It makes more mess.](https://treenodes.com/notes/ai-will-not-fix-a-messy-process/) This Markdown version is generated from the public page. Visit the source for interactive examples and full-size visuals. [Notes](https://treenodes.com/notes/) Before you add AI # AI won’t fix a mess. It makes more mess. **Fix the process. Then put AI to work.** Clear rules, trusted data and a defined task give AI a useful job, with checks and people in the right places. [Nermien Barakat](https://treenodes.com/articles/by/nermien-barakat/), Software Architect & Engineer · 2 October 2026 From the problem to the fix ![AI won’t fix a mess. It makes more mess.. By hand, an unclear step lets a person stop and ask. With AI, or software built without architecture, testing or automation, the same unclear step runs on a guess, at volume. AI does not ask unless the task tells it when to. It can sort messy information and spot inconsistencies; it cannot decide what the rule should be.](https://treenodes.com/assets/notes/slides/ai-will-not-fix-a-messy-process/slide-1.webp) **AI won’t fix a mess. It makes more mess.** By hand, an unclear step lets a person stop and ask. With AI, or software built without architecture, testing or automation, the same unclear step runs on a guess, at volume. AI does not ask unless the task tells it when to. It can sort messy information and spot inconsistencies; it cannot decide what the rule should be. ![Why it goes wrong. No rule for an invoice without an order; one supplier with three spellings; no checks, so a confident wrong guess goes straight through. If any of these is true, fix it first. The model is the easy part.](https://treenodes.com/assets/notes/slides/ai-will-not-fix-a-messy-process/slide-2.webp) **Why it goes wrong** No rule for an invoice without an order; one supplier with three spellings; no checks, so a confident wrong guess goes straight through. If any of these is true, fix it first. The model is the easy part. ![Fix the process. Connect the data. Then add AI.. Set up once: a clear rule, one supplier list, who may approve what. Then, for every invoice: the AI reads the supplier, amount and order number off the PDF, or says it is unsure; ordinary software checks the fields against the order and the delivery and routes them, a draft for approval or an exception for a person. That routing is the automation, and much of the saving. Fictional example: 400 invoices a month, approved before anything is paid.](https://treenodes.com/assets/notes/slides/ai-will-not-fix-a-messy-process/slide-3.webp) **Fix the process. Connect the data. Then add AI.** Set up once: a clear rule, one supplier list, who may approve what. Then, for every invoice: the AI reads the supplier, amount and order number off the PDF, or says it is unsure; ordinary software checks the fields against the order and the delivery and routes them, a draft for approval or an exception for a person. That routing is the automation, and much of the saving. Fictional example: 400 invoices a month, approved before anything is paid. ![One passes. One stops.. Two invoices, the same checks. Invoice A passes them all, so the system creates a draft for approval with nothing retyped; a person still approves the payment. Invoice B’s delivery does not match, 15 parts recorded as received against 20 invoiced, so it stops and goes to purchasing with the difference beside it. Passing the checks lowers the risk; it does not prove every field is right, which is why a sample of what passed is spot-checked later.](https://treenodes.com/assets/notes/slides/ai-will-not-fix-a-messy-process/slide-4.webp) **One passes. One stops.** Two invoices, the same checks. Invoice A passes them all, so the system creates a draft for approval with nothing retyped; a person still approves the payment. Invoice B’s delivery does not match, 15 parts recorded as received against 20 invoiced, so it stops and goes to purchasing with the difference beside it. Passing the checks lowers the risk; it does not prove every field is right, which is why a sample of what passed is spot-checked later. ![Match the review to the risk. Low-risk, reversible steps, such as a draft reply or fields read off a PDF, go through the checks and a spot-check of a sample later. Costly or irreversible steps, such as releasing a payment, accepting contract terms or sharing personal data, need a person’s approval before the final step, even when every check passes. Show the approver the evidence, not just a button.](https://treenodes.com/assets/notes/slides/ai-will-not-fix-a-messy-process/slide-5.webp) **Match the review to the risk** Low-risk, reversible steps, such as a draft reply or fields read off a PDF, go through the checks and a spot-check of a sample later. Costly or irreversible steps, such as releasing a payment, accepting contract terms or sharing personal data, need a person’s approval before the final step, even when every check passes. Show the approver the evidence, not just a button. ![A pilot with a pass mark. Agree the requirement and today’s numbers as the baseline. Test in a controlled environment: real cases with known answers, the awkward ones included; no live actions, and data goes only where agreed. It goes live only when it meets the pass mark; if not, fix the rule, the data or the model and test again. Live, it is measured against the baseline, spot-checked, and can be switched off.](https://treenodes.com/assets/notes/slides/ai-will-not-fix-a-messy-process/slide-6.webp) **A pilot with a pass mark** Agree the requirement and today’s numbers as the baseline. Test in a controlled environment: real cases with known answers, the awkward ones included; no live actions, and data goes only where agreed. It goes live only when it meets the pass mark; if not, fix the rule, the data or the model and test again. Live, it is measured against the baseline, spot-checked, and can be switched off. The note in 85 seconds Video · no sound The slides, plus the order to do it in, as a short silent video. The text on this page is its transcript. ## The problem Where a step is unclear, a person can stop and ask. People guess too, but the option is there. A model, unless told when to stop, guesses, confidently, and at volume, so the task has to make uncertainty visible and give the model a way to escalate. Michael Hammer warned in 1990 against automating a bad process instead of fixing it. AI makes the warning sharper. Ask AI to sort a shared inbox before anyone has agreed who handles what, and requests are routed on a guess. Ask it to read invoices when the supplier list has three spellings of one company, and duplicates appear at machine speed. The same pattern runs through a factory’s maintenance requests, a clinic’s referral letters and a building site’s daily reports: something arrives, someone reads it, decides, and passes it on. AI is good at the reading, the sorting, spotting inconsistencies and a first draft of the reply. It can even suggest a rule; your business decides which rule applies. ## Three questions first Can someone describe the process, with its exceptions and who decides? Is there an agreed, trusted source for each fact, and do you know where the data goes when a model reads it? And when the AI is wrong, how will anyone know? The last one has a trap. A model’s own confidence cannot be trusted on its own; it can be wrong and certain. A measured score, tested against cases with known answers, can help decide what gets a second look; the model’s say-so cannot. So the model does the narrow job, reading the fields off the PDF, and ordinary software checks them against the order and routes them: what passes moves on, what fails goes to a person, with the evidence beside it. Passing the checks lowers the risk; it does not prove every field is right, which is why a sample of what passed is spot-checked later. And where a mistake would be costly or cannot be undone, a payment, a contract, personal data, a person approves before the final step, even when every check passes. ## A pilot, not a leap of faith In our experience, a clearer process and connected data deliver much of the saving before any model is involved. What is left for the AI is small, testable and checked, and it is run as a pilot: a small trial on one process, for a set period, with a pass mark agreed before it starts. Agree what better means and measure today’s numbers as the baseline. Test the AI in a controlled environment, on a golden set of real cases with the answers already known, the awkward ones included, where nothing live is touched and data goes only where agreed. It goes live only when it meets the pass mark, it is measured against the baseline once live, and it can be switched off. ## How to fix it In this order. 1. Agree the requirement. What better means, the pass mark (fields right, exceptions caught, review time, cost per item), what the AI must never do on its own, and today’s numbers as the baseline. 2. Fix the process. Write down the steps, the exceptions and who decides, and the rules that never bend: the AI cannot change a supplier’s bank details; that goes through a separate, verified process. 3. Connect the data. One supplier list, one agreed source for each fact. 4. Give the AI a small task and the written rules, including when to say unsure. Ordinary software checks every suggestion against the order and routes it: what passes moves on, what fails goes to a person. Passing the checks lowers the risk; it does not prove every field is right. 5. Match the review to the risk. Costly or irreversible steps need a person’s approval even when every check passes. Show the evidence, not just a button. 6. Test in a controlled environment. A golden set of real cases with known answers and the expected action for each (continue, hold, ask a person), the awkward ones included; no live actions, and data goes only where agreed. It goes live only if it meets the pass mark. 7. Measure, then improve. Keep a record of every proposal and decision, compare with the baseline, spot-check a sample of what passed, and re-test on the golden set after any change, including a new version of the model. Which process in your business would you want clear and trusted before AI touched it? [Let’s map it](https://treenodes.com/contact/) Go deeper [Giving AI a job description](https://treenodes.com/articles/giving-ai-a-job-description/) Before AI reads an invoice or drafts a reply, decide what it should do when it gets stuck. Define the task, the review and the fallback. Sources 1. [Reengineering Work: Don’t Automate, Obliterate (opens in a new tab)](https://hbr.org/1990/07/reengineering-work-dont-automate-obliterate) Michael Hammer, Harvard Business Review, July–August 1990 2. [Humans and Automation: Use, Misuse, Disuse, Abuse (opens in a new tab)](https://doi.org/10.1518/001872097778543886) Raja Parasuraman and Victor Riley, Human Factors, vol. 39, no. 2, 1997 3. [Hidden Technical Debt in Machine Learning Systems (opens in a new tab)](https://papers.nips.cc/paper/2015/hash/86df7dcfd896fcaf2674f757a2463eba-Abstract.html) D. Sculley et al., Advances in Neural Information Processing Systems 28, 2015