Ten tests that show whether your business is ready for software to make decisions in it.

Not whether AI is good. Not which tools to buy. Whether your own records are solid enough that something acting on them would be acting on the truth.

Free, in full, with nothing to sign up for. Eight of these take under an hour. One takes a fortnight of ordinary work. None needs software, a consultant or a budget, and they work on any business with jobs, people and a system of record.

They came out of measuring one firm — an airport transfer and dispatch operation, where an AI dispatcher ran behind real controllers for seven weeks and agreed with them four times in a hundred. The book explains why that number was good news. The tests are the part you can use without reading it.

Two things to know before you start

An aggregate hides a subset. A six-centre study of an anaesthesia decision-support system reported 73.3% agreement with the clinicians. Decomposed, it was 91.1% on one drug and 17.4–29.8% on three others — and stripping that one drug out dropped the figure for everything else to 26.3%. Across the six sites the same measure ranged from 49.8% to 94.2%. The authors say it plainly: the headline "may not fully reflect the varying levels of concordance across different decision categories."

So cut any figure these tests give you by job type, by customer, by time of day, by site. One good average over four populations is the most misleading number available.

And a high number is not a licence. From the same paper: "While the AI system demonstrates high agreement with anesthesiologists, this does not necessarily validate its clinical appropriateness or safety." The threshold is set by what a wrong answer costs, not by the statistic. Nobody can tell you what is good enough except you, and test 9 is how you work it out.

Part one — can you measure anything?

1. The outward-action test

If you run only one of these, run this one.

Pick one outward action from last week — an email, a text, an invoice, a dispatch, a quote. Then answer three questions using only your records:

Did it happen? Not was it triggered. Did the thing go out.
To whom? The address it actually reached, not the one on the customer record today.
Did it land? Delivered, opened, received, signed.

The rules are the test. No asking the person who sends them. No admin screen showing a flag. No it must have gone, we'd have heard. Records only — because records are the only thing an AI layer will ever have.

Twenty minutes.

A bad result: you answer question one and stop. Or you answer all three, but only inside your email provider's dashboard — which means the knowledge lives somewhere you don't control and can't join to anything.

To fix: one table, one write, per flow. What was attempted, to whom, whether it landed. A day or two for the first, less for each after. It is not a project and should not be allowed to become one.

2. The counter test

Find a field that looks like it records a success — sent, notified, delivered, processed. Then make it fail on purpose. Send to a dead address. Trigger the action in a way you know cannot complete.

Watch the field.

Ten minutes, plus finding somebody willing to break something.

A bad result: the field moves anyway.

Why that is worse than a gap. A blank field is honest — it tells you nothing and announces that it is telling you nothing. A field that moves on failure is false information with an audit trail, and it will be believed for years, because it behaves exactly like a working counter. Dashboards built on it are wrong and green. An AI layer reading it will report, fluently and with a citation, that something happened to a customer who was never contacted.

To fix: an afternoon per field. Set the flag from the result, not the attempt. The hard part is finding them, which is what this test is for.

3. The flag-accuracy test

Your business holds flags that stand in for documents: insurance valid, licence current, certification in date, training complete, contract signed. Take five records the system says are compliant and open the five underlying documents. By hand. Today.

Forty minutes.

A bad result: any of the five disagrees. In the firm this came from, the flags were wrong for 260 of 1,252 active drivers — and 178 of those held a document that had expired while the system showed them clear.

Check the direction of the error. A record showing not valid when it is valid gets fixed within the hour, because somebody complains. A record showing valid when it is not generates no complaint from anyone — which is exactly why it survives for years.

To fix: the audit is an hour. The discipline is a week — what keeps this flag true? An expiry date, a reminder, an automatic block. If the answer is "somebody updates it", you have found a copy that will drift.

Part two — can you see what the machine will see?

4. The impossible-value test

The cheapest finding available anywhere on this page.

Take the field your automation most depends on — a capacity, a weight, a duration, a rate, a size. Sort it ascending and read the first twenty rows. Sort it descending and read the last twenty. A spreadsheet export is fine.

You are looking for physically impossible values, not unusual ones: the zero, the one, the 9999, the 1970 date, the eleven-second job. Impossible is a judgement you can make instantly and without argument.

Then divide the impossible rows by the total.

Ten minutes.

A bad result: anything over about five per cent — or any impossible value at all in a field that gates a decision. In the firm this came from, eleven vehicles were recorded as carrying zero passengers and sixteen executive saloons as carrying one. That one field caused 77 of 125 allocation refusals. The software was not confused. It was correctly reading a wrong world.

To fix: under one per cent, by hand this week. Over five, you have found what to do before buying anything.

And refuse the advice that comes next. When somebody tells you to clean your data first, decline — it is unbounded, it absorbs any budget, and it is a direction rather than a finish line. The bounded version is: find the three fields your automation will actually read, and fix those.

5. The writers test

Name the single most important field in your business — the one whose value you would least like changed without your knowing. List every route by which it can change.

Candidates: the current system, the old system nobody retired, the admin panel, a spreadsheet import, the support tool, a partner's integration, a scheduled job, and whoever still has a password from 2019.

Fifteen minutes, longer if you are honest.

A bad result: more than one entry — and worse, an entry that skips the rule you believe is enforced. In the firm this came from, one of three writers bypassed the eligibility check entirely, so the control preventing unqualified people from working was a control on two of the three paths.

A gate is not a gate until you have listed every writer. Your rules exist only where they are checked.

To fix: consolidating writers takes months and is usually the wrong first move. The cheap version is to make every writer emit the same record, then close the one route that skips the rule you care about most. One route, this month.

6. The attribution test

Pick something that matters and definitely changed recently — a price, a booking time, a stock level, a permission. Find out who changed it, when, and from what to what. Through the product. Not by asking a developer, not by asking the office.

Give yourself ten minutes. The limit is part of the test: an answer that takes a day to assemble is not one you will ever get during an incident.

A bad result: a shared login, a blank, a timestamp with no actor, or it must have been the sync.

Why this is a precondition rather than a nicety. A firm without a service identity has two options when it automates: let the software act as a member of staff, or not automate. Almost everyone picks the first without noticing they picked anything — and from that moment the audit trail names a person who was asleep.

Then the half nobody tests. Take the last three things your automation did and ask: can a human undo this from the screen, at four o'clock on a Friday, without a developer? If not, you are not supervising it.

To fix: a week, mostly convention rather than code.

7. The exceptions test

The only one that needs another person, and the only one that will make somebody feel useful.

Ask the longest-serving person on your operations desk:

Which records do we all know to treat differently?

Write down what they say. Don't argue, don't tidy, don't stop at three — the first three come immediately and the useful ones come at minute twenty. Ask a second person separately, and where the two lists differ you have found something better than an exception: a practice two people do differently and both believe is the rule.

Half an hour, over tea.

A bad result: a long list, with none of it written down anywhere. In the firm this came from, six account records were payment rails rather than customers — and forgetting that once produced a £2.18 million invoice across 23,890 already-paid journeys, with nothing malfunctioning at all. Every component did its job on rows that were not what the table said they were.

Why it matters more than it sounds. An AI layer's entire value is applying the general rule at volume. These are the rows where the general rule is wrong, and the machine will treat them as whatever the schema claims.

8. The egress test

Answer four questions about personal data leaving your business for any AI or third-party service: what fields, to whom, how often, retained how long.

Then the harder one: if a customer asked what had been done with their data, could you answer for them specifically — not for the volume in general?

An hour, plus however long the provider takes to reply, which is itself a finding.

A bad result: not knowing the retention period, or being able to say how many records were read but not whose.

To fix: the masking belongs at the boundary, decided once, rather than per feature by whoever ships next. Asking the provider is an email — and if they cannot answer plainly, you have learned the most useful thing available about them.

Part three — should it act, and how often is it right?

9. The cost-of-error ranking

Two columns. On the left, every place software could plausibly act in your business. On the right, not a score out of ten — the actual consequence of being wrong once: a redraft, a phone call, a refund, a lost customer, a regulator's letter.

Sort by the right-hand column and build from the cheapest end.

Twenty minutes.

A bad result: your current plan starts at the expensive end. It usually does, because the expensive surfaces are the impressive ones. The most impressive surface is the one to build last.

Three checks while the paper is out:

Count your overlapping tools, not your tools. More than fifteen well-defined, distinct ones can work; fewer than ten that overlap can fail. Ambiguity is the defect, not breadth.
The whiteboard test. Take one real situation and ask the people who built the system which tool should fire. If they hesitate, the model has no chance — and no upgrade fixes a problem that lives in the design.
Count what you have built against what runs. In the firm this came from, two of eleven applications and three of twelve services served a real request. Nobody had ever put those two numbers side by side.

And before you build a registry, a gateway or a second agent: count your services. That estate had seven. The literature recommending those layers is written by people with ten thousand.

10. The agreement test

The one that takes a fortnight, and the only one that measures what you actually want to know.

Pick the one decision your software makes that you would defend most confidently. For the next twenty of those: before you look at what it suggests, write down what you would have done. One line, on paper. Then look.

Two weeks of ordinary work, no setup.

This has a proper name. Medicine calls it a silent trial and the figure concordance. Credit and insurance measure the mirror image and call it the override rate — where a rate above a ceiling is treated as a signal that the model needs rebuilding, not that the staff need retraining. Be warned that neither field has settled guidelines: a 2026 scoping review in Nature Health screened 891 papers, included 75 silent evaluations from 2015 to 2025, and states that there are no formal guidelines for performing silent AI evaluations in healthcare settings. It also found that those studies overwhelmingly measured technical performance, and that far fewer checked their outputs against a real-world ground truth — which is test 3's problem, arriving in a hospital.

If it agrees with you most of the time: good, cheaply learned. Widen what it is allowed to do, one job class at a time, never globally.

If it disagrees most of the time: do not reach for a better tool. Take five of the disagreements and write down what you knew that it did not. That page is the most valuable document in your business, it does not exist yet, and it will not run past one side.

Two refinements that matter more than the headline. Decompose it — the firm this came from measured 4.1% across 2,939 allocations and still has not cut that by job type, and one live possibility is that the system is nearly right about one class of work and hopeless about four. And exclude the cases where the machine was right: part of that disagreement was the system correctly refusing people whose paperwork had lapsed, against a flag that said they were clear. Those are logged as the machine being wrong. Leave them in and you train the next version to repeat the error.

If you cannot run this test at all — because your system records what was done but not what it suggested — that is the finding, and it is test 1 again.

Scoring, and the honest reading

Count the tests you pass. Pass means a clean result, not one you can explain away.

Nine or ten. You are better instrumented than the firm these came from. The agreement test is now the interesting one, and you can widen authority on evidence rather than hope.

Six to eight. The normal good case. Fix the ones that gate decisions — 1, 3 and 6 — then start at the cheap end of test 9.

Three to five. You have enough visibility to know you do not have enough. Do not buy an agent. Do tests 1 and 2 properly on your three most important flows and run this list again in a quarter.

Nought to two. Where the firm in the book sat, and worth saying plainly: not a crisis, and not unusual. That business was running profitably, serving real customers, with a working app and a desk that knew its job. An open loop is not a failing business. It is an invisible one, and the cost is that you cannot be told you are wrong.

In every band, the same instruction: run tests 1 through 8 whatever you decide about AI. They are not AI preparation. They are the questions a well-run business can answer about itself, and every one was worth asking before any of this arrived.

These ten tests are the appendix of Shadow Mode, and they are here in full because they are more use to you than to me. The book is the other part — ten chapters on where each of these came from, measured inside one firm on dated days, including the ones that were embarrassing. Nothing on this page is a sample of it; this is the whole appendix.