Rules or AI? When a plain decision tree is enough

When a company is deciding whether to add AI to its system, it usually asks which model to choose. It is better to start with a different question: would plain rules do the job? We tried this in the Expiro app. We gave rules and an AI model the same task on 425 situations, and the result surprised us. Neither approach won. Both together worked best.

An angry customer does not write

Expiro is a service that keeps track of company deadlines for technical and emissions inspections (STK and EK), compulsory motor insurance and motorway vignettes. The company enters its licence plates, Expiro looks up the dates in the register on its own and sends a reminder before each one.

Picture a fleet manager who opens Expiro and sees that one van’s technical inspection is “overdue”. Yet they took it for inspection last week. They get annoyed.

But they will not write to support. They do not feel like looking up an email address, describing the problem and waiting for a reply. They close the app and may stop using it. The operator never finds out what happened, and they remember that “the app doesn’t work”.

Four suggestions instead of an empty box

So we added a “Niečo nesedí?” (“Something wrong?”) button to the app. Clicking it does not open an empty box to type into. The customer immediately sees four things that are probably bothering them, put together from their own vehicles and from what they were just doing in the app.

They click on their problem and immediately get an explanation and a button that fixes it. For a wrong date, for example, it is “Enter the inspection date manually”. In ten seconds it is sorted, and they did not have to type anything. Whatever cannot be fixed this way reaches support already sorted, together with the vehicle and the context.

The whole thing rests on one ability: guessing the right four options. There are more than fifty prepared, from a wrong date through a broken invitation to the question of whether the service is worth it. If customers do not find their problem among the four suggestions, the button is useless to them.

First attempt: plain rules

The simplest approach is to write rules, that is, conditions of the type “if this is true, show that”:

• The vehicle’s technical inspection is overdue, so the customer may think the date is wrong.
• They clicked “Check again” three times in two minutes, so this button is probably what is annoying them.
• Their card payment failed, so it is almost certainly about the payment.

Programmers call this a decision tree. It is fast, costs nothing, and you always know exactly why it decided the way it did. When the customer types nothing and just clicks the button, rules work very well. The app can see what is happening to them right now.

Where rules hit the wall: people write their own way

The customer can, however, also click “Something else, I’ll type it” and write, for example (customers wrote in colloquial Slovak, often without diacritics; the quotes below are translated in the same spirit):

we were at inspection yesterday and its still showing red on your end

A rule only knows the words someone gave it in advance, such as “STK”, “technical” or “we did the inspection”. The customer wrote “inspection” and “showing red”, without diacritics and in their own words. A person knows at once what they mean. A rule does not.

You can add more words. But you will never think of every way people will phrase it, and with every new problem on the list you have to start thinking again. Among the texts we had written in advance, these are some the rules could not make sense of:

• “We handed this car back ages ago, why do you keep showing it here, fix it already!”
• “Could you send it to my fone, I often dont see email in time”
• “Not sure it’s still worth it for me with only a few vehicles, maybe I’ll switch to excel.”

Second attempt: AI that only chooses

For the texts we deployed the Jev model from TypeSafe. What matters is what it does not do. It is not a chatbot like ChatGPT and it does not write any replies. It can only choose from the options you give it and say how confident it is.

It gets two things:

1. A description of the situation: “The customer has the free plan. One vehicle’s technical inspection is overdue. They clicked Check again twice. They wrote: …”
2. A list of options: more than fifty prepared problems.

It returns numbers, for example: “Inspection date is wrong: 99%, Check again does not work: 1%…” It understands the sentence about the inspection and the red colour just as a person would, even without diacritics and with typos.

Why a model that only chooses? Because it cannot make anything up. Every text and button the customer sees was written by us. The AI only decides which of them to show first. It cannot promise the customer anything or give them wrong information, which is something you cannot guarantee with an ordinary chatbot.

The test: who guessed better

Before launch we had both approaches handle the same situations, where we knew the right answer. We counted how often the customer’s real problem was among the first three options offered. For comparison we also added the simplest option: the same list for everyone.

Each approach is strong where the other is weak:

• Rules win when the customer types nothing. They see exactly what happened in the app, and AI adds nothing there.
• AI wins when the customer writes. It understands words we could not foresee.
• The same list for everyone is almost useless. The customer would find their problem in one case out of ten.

That is why Expiro uses both at once. Every option is scored by both the rules and the AI, the scores are averaged and the top four are shown. When a rule is completely certain, for example with a failed payment, that option is always shown.

One opening of the window costs roughly a hundredth of a cent ($0.11 per thousand openings), and the AI usually answers within a third of a second. How we put the test together so the numbers would not mislead us, and everything we measured, is in the article how to test AI before your customers see it.

Safeguards without which we would not have launched AI

• AI does not decide important things. Whether a vehicle is overdue is always calculated by an ordinary program. The AI only estimates what is bothering the customer.
• AI does not see personal data. We do not send it licence plates, names or email addresses. It gets the vehicles as “vehicle 1” and “vehicle 2”.
• If the AI does not answer within 2.5 seconds or is not working, the rules put the list together on their own. The customer notices nothing.
• The customer always triggers the action. The model only orders the options. It does not delete, change or send anything by itself.
• Serious matters go to a person. When a customer writes that they got a fine even though Expiro showed “OK”, no automatic reply is sent. A person gets in touch within 24 hours.

So a plain tree is not enough?

Honestly: often it is. When your customers only click and you know what is happening to them in the app, well-written rules are faster, cheaper and just as accurate. Roughly half of our fifty problems are recognised by a plain rule, because they have one clear signal. You do not need AI there, and nobody should be selling it to you.

AI pays off where all of the following hold:

• People write in their own words and there is no way to predict in advance how they will phrase it.
• The answer is a choice from fixed options, so the AI has no room to make anything up.
• A mistake is not a disaster, because the customer always sees several options and chooses themselves.
• There is a fallback that works without AI.

In Expiro all of these hold. That is why we did not build AI instead of the rules, but alongside them. We use the same principle in customer support automation and in email sorting: rules first where they are enough, and AI where human language begins.

What we do not know yet

The test texts were not written by real customers. They were written by another AI model based on short stories, so that it would not use the same words as our rules. Only live operation will show the real numbers. So half of the companies get suggestions from rules only, and half from rules with AI. Every week we compare which group more often hit what the customer actually wanted.

When we have results from live operation, we will write a follow-up, even if they turn out differently.

Summary

• Rules are excellent at what they were written for: clear signals in the app. On words their author did not foresee, they drop from 97% to 72%.
• AI that only chooses from fixed options understands colloquial language and cannot make things up.
• Both at once is best, with a fallback and a person for serious matters.
• Before you buy AI, ask whether a rule would do. Often it will.

Working through a similar question in your own system? In an AI audit we will tell you where rules are enough and where AI really pays off.