Measuring AI literacy: why a quiz isn't enough

Knowing isn't doing

Many AI courses end with a quiz: what is a language model, what does the AI Act say, what is a hallucination. Whoever passes knows the answers. But mistakes at work happen elsewhere: a customer file pasted into a free chatbot, a VAT amount from an AI summary nobody checks, a vague instruction that costs half an hour instead of saving five minutes.

Someone can explain perfectly what a hallucination is and still copy an invented invoice number because it looks convincing.

What to measure instead

AI skill shows in situations that resemble real work. For example: which of these data may go into which tool? Does this AI summary of an invoice match the source? Which of these three instructions produces a usable customer email, and why? What do you do when the AI gives an answer that contradicts the work rules?

Tasks like these measure judgement, not memory. They show whether someone checks, spots sensitive data and can give AI a clear instruction.

Per department, not per person

For a company, the most useful result isn't one employee's score but the pattern per department. Accounting may check AI figures well while sales pastes customer data into the wrong tool. Then you know where a short lesson pays off most.

Measure twice

A first measurement shows where you stand. After a targeted lesson, a second measurement with different tasks at the same level shows what changed. That difference is also the strongest evidence that your measures under Article 4 of the AI Act worked.

Try it yourself

On AISkillProof you can do a short exercise without an account and see what such a work task looks like. It isn't a scored test, but it gives a good sense of the difference with a quiz.