How to Choose Score Weights for a Recommendation Quiz

Keep firm requirements outside the tradeoff

An additive score permits compensation: enough points from one answer can offset fewer points elsewhere. Use that only where the substitution makes sense. Wanting more feedback might justify a guided format despite a mild preference for independent study. It cannot make someone available for a session they cannot attend.

Separate requirements from preferences before assigning weights. A requirement decides whether an offer is usable at all; a preference helps choose among usable offers. Check requirements explicitly in the decision flow. A large negative score is a fragile substitute because later questions or changed weights can accidentally cancel it.

Treat an unknown requirement as unresolved, rather than assuming it passes. If no available offer meets both the requirements and the person's needs, explain the missing fit or offer clarification. Sending someone to the remaining offer merely because it is available can still produce poor advice.

Weight the difference between answers

Write down the least and most consequential answers for each criterion. Then ask how much moving between those endpoints should change the recommendation. This keeps the weight tied to an actual difference, rather than a vague statement that a topic is important.

UK government guidance on multi-criteria decision analysis makes a related distinction: weights consider the difference between the options and how much that difference matters. Its formal method is designed for options appraisal. The practical lesson for a small quiz is narrower: define the answer range before judging its influence.

With answer points, compare each question's largest possible contribution minus its smallest. A question ranging from zero to four can change the total twice as much as one ranging from zero to two. A question whose answers all receive five points cannot distinguish respondents, although it shifts every total relative to a fixed cutoff.

Use simple whole numbers while testing. More digits do not supply more evidence. If the team cannot explain why a particular answer should outweigh another, resolve that disagreement in words before encoding it in points.

A worked example: workbook or guided clinic?

Consider a hypothetical creator teaching people to build a presentation. The offers are a self-paced workbook and a live small-group clinic. Both teach the same task; the clinic also provides planning help, supervised practice and individual feedback. The quiz is meant to recommend the amount of support, not measure talent or willingness to spend.

The creator starts with three single-select questions. Each asks about a different obstacle in completing the task. These are trial weights and boundaries for this example, not validated assessment scores.

The possible total is zero to eight. The creator provisionally considers the workbook for zero to two, asks a clarifying question at three, and considers the clinic for four to eight. The reasoning is that a need for full draft review, or combined needs for planning and supervised practice, calls for support the workbook does not provide. Whether those boundaries work still needs testing.

Someone scoring four for feedback and zero elsewhere reaches the same total as someone scoring zero for feedback and two on each other criterion. The clinic may suit both because it actually provides both kinds of help. But their explanations should differ: one needs draft review; the other needs planning and supported practice. A total alone cannot recover that distinction.

Live attendance is checked separately. If someone needs clinic-level support but cannot attend, the flow should not recommend the workbook as an equivalent replacement. It can explain the limitation and offer a way to ask about another format, provided the business can handle that request.

Look for duplicated influence and lost meaning

Read the questions together. Asking whether someone wants expert feedback, a personal review and comments on their draft may give the same need three opportunities to add points. The European Commission's Joint Research Centre warns that related indicators can introduce double counting, and that their meaning matters alongside any statistical relationship.

For a small quiz, explain what new information each question contributes. In the example, planning the sequence, practising the steps and reviewing the draft are different tasks. If audience testing shows people interpret all three as the same request for help, rewrite or combine them. Similar answers alone do not prove duplication; the issue is whether the questions add distinct decision evidence.

Also inspect different answer combinations that produce the same total. If they require different offers, the model has discarded a distinction the business needs. A larger weight may separate two cases temporarily while creating collisions elsewhere. Use a direct rule or another question when the distinction itself determines the next step.

Test the model before trusting the number

Write a few contrasting answer profiles and decide the appropriate next step before calculating their scores. Include someone who clearly needs little support, someone who clearly needs the clinic, mixed needs, an unknown requirement and someone unable to attend. Expected advice should come from the offer's actual capabilities, rather than whatever the formula happens to return.

Test the proposed boundary directly. In the example, compare totals of two, three and four, and inspect every combination that reaches them. At three, the follow-up must resolve a useful distinction: does the person need help while doing the work, or can a written example address the uncertainty? If neither answer changes the advice, the follow-up has no useful job.

That boundary test exposes a problem in the trial model. Someone who needs another person to plan the work scores two even with zeros elsewhere, yet the workbook cannot provide that help. The creator adds a direct rule for essential planning or supervised-practice needs, as well as full draft review. Those answers require checking clinic fit regardless of the total. The remaining score can guide clarification about flexible preferences; it must not overrule an essential need. If these direct questions settle every useful distinction, remove the score entirely.

Then hold the answers fixed and vary a weight you are unsure about. For instance, reduce the maximum feedback contribution from four to three while leaving the other points and trial boundaries unchanged. In the score-only trial, the draft-review profile now enters clarification instead of the clinic. That reveals a consequential assumption: whether this need alone should trigger a clinic recommendation. The direct rule protects that essential need in the corrected flow. Use the same test on the remaining preferences to find other fragile assumptions.

This is a small sensitivity test: checking how a result responds to a modeling assumption. The Joint Research Centre describes sensitivity analysis as a way to identify influential assumptions in composite measures. Stability is useful evidence about the model; it does not prove the recommendation is right. A model can be consistently wrong.

Record which profiles change and why the change is acceptable or unacceptable. Fix the underlying criterion, question or decision rule, then rerun the contrasting profiles. Avoid tuning points until a few favourite examples work while leaving the rest unexamined.

Implement the scoring you actually designed

In QuizFlow Labs, Scoring Logic lets you assign answer points to Multiple Choice questions. Use single select for the example's one-answer-per-criterion design. Multi-select sums the selected option scores, so changing selection mode can also change a question's maximum influence. Recalculate the range rather than carrying over the old weights.

Keep eligibility decisions explicit with routing or appropriate answer-based outcomes. Score-based Conditional Outcomes compare a total with numeric conditions; they do not by themselves establish whether a particular offer meets a requirement. Configure the requirement path and test its final message and action as well as its questions.

Within score conditions, the first matching condition wins. A between range includes both endpoints, so adjacent ranges that both include four overlap at four. Use non-overlapping boundaries and check the default outcome too. Variable-match conditions are evaluated before intent conditions, which precede score conditions; inspect any existing overrides before interpreting a surprising result.

Test each relevant route's attainable totals. A path that skips a scored question may no longer have the same possible range as a path that asks it. Avoid interpreting a smaller total as lower need when the person simply had fewer opportunities to contribute points. Use comparable scored questions or separately justified route-specific decisions.

Diagnose wrong recommendations before changing weights

When advice looks wrong, compare the selected answers, their intended contributions, the calculated total and the condition that selected the outcome. A wrong total calls for checking configuration or the scored route. A correct total with the wrong result calls for checking boundaries and higher-priority conditions. A correctly applied rule that gives poor advice calls for revisiting the decision model.

Watch for one question deciding nearly every result, several questions repeating the same need, or equal totals requiring incompatible advice. These patterns suggest different remedies: reduce unjustified influence, remove duplication, or preserve an important distinction outside the sum. Multiplying every point value by the same positive factor leaves relative influence unchanged; if numeric boundaries are multiplied by that factor too, the score-based decisions stay the same.

After launch, review whether recommendations match the needs expressed in the answers and what the offer can deliver. A booking or purchase shows an action occurred; it does not establish that the advice was suitable. Use appropriate follow-up about unmet needs or confusing guidance before treating conversion alone as a reason to change weights.

Start with a clear score meaning, a small set of distinct criteria and explicit requirements. Keep the contrasting profiles as regression examples whenever questions, weights, offers or outcome rules change. The aim is advice you can explain and defend, with numbers that serve that decision.