What is the best scoring algorithm for compliance training?

MDT, with guessing correction on and a wrong-answer penalty around 0.25. Compliance results are often used as evidence, and a plain percentage lets a lucky guess look like knowledge. Keep the penalty out of practice rounds and apply it to the graded round.

When is a simple percentage good enough?

When your questions are close to even in difficulty, the audience needs a number they can interpret without explanation, and guessing is not a serious risk. It is also a sensible starting point for a first programme before you have difficulty data.

Should I score difficulty-weighted or penalise wrong answers?

They solve different problems. Difficulty weighting fixes an uneven question bank; penalties fix guessing. If both apply — a mixed bank where wrong answers matter — MDT does both, because it weights by difficulty and subtracts for incorrect answers.

How does question selection change the score?

Selection decides which questions a person sees and when. Ask everything once and you measure recognition on the day. Space the same questions over weeks and you measure retention. Two people can answer identically and end up with different scores simply because one was asked at spacing intervals and the other in a single sitting.

Can I change the method after a programme has started?

Yes. Scoring is recalculated from the stored answers, so changing algorithm changes the reported score without losing history. Changing the selection method affects future rounds rather than past ones.

Does this tool store my answers?

No. The recommendation is computed in your browser and nothing is sent to a server. Copy or download it if you want to keep it.

Free tool

Assessment Method Picker

Pick the right scoring algorithm and question selection method for your material. Answer five short questions about what you are teaching and how you will use the result — the recommendation updates as you go.

Why the method matters

The same set of questions can produce very different numbers depending on how you score them and how you choose which question to ask next. A percentage score is easy to read but treats a hard scenario question exactly like a one-word recall item. A mastery rule rewards repeated success but says little after a single sitting. Selection matters just as much: cramming everything into one session measures short-term recall, while spacing the same questions over weeks measures what people retain. Answer the questions below to get a starting point you can adjust later.

Tell us about your content

What is the material?

What is the goal?

What are the questions like?

Do you have easy and hard questions mixed together?

Do you need to discourage guessing?

Nothing you answer is sent anywhere — the recommendation is calculated in your browser.

Your recommendation

Scoring algorithm

Progressive mastery

A run of consecutive correct answers, not a single hit.

Runner-up: Simple percentage Questions of roughly even difficulty where the number has to be obvious to everyone.

Question selection method

Memorization

Short, frequent rounds that keep resurfacing what is still unseen, wrong or answered without confidence.

Runner-up: Spaced repetition Anything people must still know months later.

Why this fits

You care about what people still know later, so the score should reward repeated correct answers rather than one good day.
Terminology, product facts, drug names, part numbers.

Suggested starting settings

Consecutive correct required3

Watch for interference

Your content has closely related concepts that compete with each other in memory. Vary the question format within a topic, keep confusable items apart in the same session, and track mastery per topic rather than per programme so you can see which pair is being mixed up.

Scoring algorithms compared

Method	What it rewards	Use it when	Avoid it when
MDT (measurement decision theory)	Difficulty-weighted correct answers, minus a penalty for wrong ones.	Compliance, safety and certification, where a wrong answer is not neutral.	Early practice, where penalties discourage attempting anything.
Weighted difficulty	Harder questions are worth more than easy ones.	Banks that deliberately mix easy recall with hard scenarios.	When difficulty ratings are unreliable or missing.
Progressive mastery	A run of consecutive correct answers, not a single hit.	Facts, terminology and procedures that must become automatic.	One-off exams where a single sitting is all you get.
Adaptive performance	Correct answers on questions above a difficulty threshold, with a bonus.	Stretching strong performers without punishing the rest.	Small question banks where the bonus swings the score.
Time decay	Recent answers more than older ones.	Measuring whether a skill has decayed since the last round.	First-time baselines, where there is no history to decay.
Simple percentage	Correct answers out of the total, nothing else.	Questions of roughly even difficulty where the number has to be obvious to everyone.	Mixed difficulty, or anywhere guessing is cheap.
Custom formula	Whatever your formula says — correct answers, streaks, confidence and difficulty are all available.	An existing internal scoring rule you have to reproduce exactly.	Anywhere a standard algorithm already fits, since a formula is harder to explain.

Selection methods compared

Method	What it rewards	Use it when	Avoid it when
Test prep	Coverage before a known date, working through phases as the date approaches.	Certification and exam preparation with a deadline.	Open-ended, continuous training with no exam.
Memorization	Short, frequent rounds that keep resurfacing what is still unseen, wrong or answered without confidence.	Terminology, product facts, drug names, part numbers.	Judgement-heavy material where the answer depends on context.
Spaced repetition	Long-term retention, expanding the gap between successful repeats.	Anything people must still know months later.	A single assessment sitting with no follow-up.
Reassessment	A fresh measurement on the same competencies after a period of time.	Annual refreshers, decay checks and audit evidence.	First-time learners who have not been measured yet.

How the two choices interact

Scoring and selection pull in different directions if you choose them separately. A mastery score needs a selection method that returns to the same question more than once, so it belongs with memorization or spaced repetition rather than a single sitting. A penalty-based score such as MDT is fair at the end of a programme but harsh during early practice, so pair it with a selection method that gives people repeated exposure before the graded round. And a time-decay score only means something if the selection method actually spreads answers across weeks. As a rule: pick the selection method from your goal and timeline first, then pick the scoring algorithm that matches the question bank you have.

Run it in Metronic

Metronic implements every scoring algorithm and selection method on this page. Upload your material, generate questions, and apply the recommended combination without building any of it yourself.

Start free — 200 credits Read the spaced repetition guide

Frequently asked questions

What is the best scoring algorithm for compliance training?: MDT, with guessing correction on and a wrong-answer penalty around 0.25. Compliance results are often used as evidence, and a plain percentage lets a lucky guess look like knowledge. Keep the penalty out of practice rounds and apply it to the graded round.
When is a simple percentage good enough?: When your questions are close to even in difficulty, the audience needs a number they can interpret without explanation, and guessing is not a serious risk. It is also a sensible starting point for a first programme before you have difficulty data.
Should I score difficulty-weighted or penalise wrong answers?: They solve different problems. Difficulty weighting fixes an uneven question bank; penalties fix guessing. If both apply — a mixed bank where wrong answers matter — MDT does both, because it weights by difficulty and subtracts for incorrect answers.
How does question selection change the score?: Selection decides which questions a person sees and when. Ask everything once and you measure recognition on the day. Space the same questions over weeks and you measure retention. Two people can answer identically and end up with different scores simply because one was asked at spacing intervals and the other in a single sitting.
Can I change the method after a programme has started?: Yes. Scoring is recalculated from the stored answers, so changing algorithm changes the reported score without losing history. Changing the selection method affects future rounds rather than past ones.
Does this tool store my answers?: No. The recommendation is computed in your browser and nothing is sent to a server. Copy or download it if you want to keep it.