Should a Customer Survey Use Agree/Disagree Statements?

Start with the decision, then name what you need to measure

Write down the change you might make before writing the survey item. “Improve onboarding” is too broad. “Decide whether to simplify the first setup task” gives the question a specific job.

Next, name the thing you need to learn: difficulty, frequency, satisfaction, confidence, amount or preference. These are different measurements. Satisfaction with a lesson does not establish how often someone used it. Confidence about a task does not demonstrate that they can perform it.

For the setup decision, “How easy or difficult was it to complete the first setup task?” is more informative than “The onboarding was good.” Use answer labels that describe difficulty, and specify which task and occasion the person should consider. Someone who never attempted it needs an answer outside that difficulty scale.

A direct question cannot rescue a vague decision. If no possible answer would affect what you do, remove the item or explain the separate learning purpose it serves. Do not collect a rating simply because the questionnaire feels incomplete without one.

Understand what changing the format can fix

An agreement statement presents an assertion for the respondent to accept or reject. That creates a risk of acquiescence: agreeing with the assertion regardless of its content. Pew Research Center discusses this risk in its questionnaire-design guidance. It is a reason to examine the format, not a reason to dismiss every positive answer as insincere.

Direct questions also let the answer labels describe the experience itself. “In how many of the last four weekly sessions did you use the planning template?” can offer zero, one, two, three or four sessions. Agreement with “I regularly use the template” leaves each person to decide what regularly means.

There is a cost: each question needs its own suitable labels, so a copied set of agreement choices will no longer serve every item. That extra design work is worthwhile when the distinction changes the decision. It is unnecessary to create a detailed scale for information you only need as a yes or no.

Do not try to solve vague wording by adding more agreement points or by inserting awkward negative statements. “I did not find the task unhelpful” asks the respondent to untangle the sentence before answering. A plain question about usefulness is easier to discuss and test.

Match the answer scale to the possible action

A difficulty scale answers how hard something was. An amount scale can tell you which direction to adjust. Choose between them deliberately. If the question is whether to increase or reduce contact, “far too little, a little too little, about right, a little too much, far too much” preserves the two directions that a single unfavorable rating would combine.

The NSW Department of Education recommends response options specific to what is being measured rather than relying on agreement labels. Its guidance also distinguishes a meaningful midpoint from “don’t know.” These are useful distinctions for customer feedback: “about right” is an evaluation; “I did not use this service” is a different condition.

For satisfaction, give dissatisfied and satisfied respondents comparable ways to express intensity, with a neutral position if it is meaningful for the question. For a count of uses, begin at zero and use actual counts when people can reasonably recall them. There is no need to force a satisfaction-style midpoint into a count.

Label each choice in words or meaningful quantities instead of expecting people to infer what an unlabeled number means. Keep ordered labels in order. Check that the question and every label form a sensible answer together, and that two choices do not describe the same position.

A worked example: deciding whether to change office hours

Consider a hypothetical membership creator deciding whether to change the number of live office-hour sessions. The original item is “I am satisfied with the office hours,” followed by agreement choices. Even a clear unfavorable response would not show whether session quantity is the problem.

First establish exposure: “How many of the four office-hour sessions offered in September did you attend?” The choices are zero through four. Someone selecting zero has no attendance experience from that month, although they may have useful information about access or timing.

For attendees, ask “For the help you needed in September, how was the number of office-hour sessions offered?” Use “far too few,” “a little too few,” “about right,” “a little too many” and “far too many,” plus “not sure.” This asks about the number offered, not the number attended. Keep that distinction explicit.

Then invite an optional explanation: “What, if anything, would you change about the office hours?” An attendee might value the number offered but struggle to get a question answered. Adding sessions is only one possible response; changing the format could address a different problem.

For non-attendees, ask what kept them from attending rather than requiring a session-quality rating. Scheduling, no current need and not knowing the sessions existed call for different investigations. Test these categories with members before using them as a fixed answer list; leave room to describe a different reason.

This design costs more than one broad rating. Use it when the session decision matters enough to justify the extra questions. If the purpose is only to track overall satisfaction, keep a direct satisfaction question and avoid pretending it diagnoses the cause.

Keep uncertainty out of the middle of the scale

A neutral evaluation, a mixed experience and insufficient knowledge are not interchangeable. Someone may judge the amount of support to be about right. Someone else may have used it too little to judge. Combining both in the middle makes the result hard to interpret.

Offer “not sure,” “did not use” or “not applicable” only where each represents a real possibility, and make their meanings distinct. A separate exposure question can be clearer than a long list of exceptions attached to every rating. Avoid asking people to rate an experience they have not had.

In reporting, show how many people were able to evaluate the experience as well as their answers. If you report the share saying “too few,” state that the denominator is the attendees who answered that item and explain how uncertain answers were handled. Do not quietly count non-use as a neutral or negative score.

Know when to retain an agreement statement

Agreement can be appropriate when the assertion itself is the subject: for example, whether members support a proposed community rule. State the proposal precisely, and give support and opposition an equally clear expression. A direct support/oppose question may still be the simpler choice.

Be more cautious when changing an established questionnaire used for comparisons or a validated multi-item instrument. Editing one statement, its scale or its scoring can change what the resulting measure means. Follow the instrument’s documentation rather than treating a rewrite as a cosmetic improvement.

For a homegrown customer survey, record the wording change and treat the revised question as a new baseline unless you have evidence that the measures are comparable. A lower rating after a rewrite does not by itself show that the service deteriorated. Do not compare an agreement percentage with a satisfaction percentage as if only the date changed.

Test interpretation, then diagnose the response pattern

Before sending the revised survey widely, ask a few people from the intended audience to answer it and explain how they chose. Ask what experience they considered, what each end of the scale meant and whether their answer fit a label. Include someone who has not used the service if that is a plausible respondent.

If people interpret “support” as staff help, peer discussion and written instructions, narrow the question to the service you can actually change. If they agree on the experience but cannot distinguish adjacent labels, simplify or clarify the labels. If they lack an answer because they did not participate, repair the exposure path rather than forcing a judgment.

After launch, investigate patterns before turning them into actions. Many “not sure” answers may reflect vague scope or limited experience; they do not prove indifference. Responses at both ends may signal different needs; an average near the middle can hide that split. Similar ratings across every item may reflect genuinely similar experiences or a questionnaire people answered with little differentiation. The pattern alone cannot identify the cause.

Use short follow-up conversations or concrete examples to distinguish these explanations. Choose the next business change from the combination of the rating, its context and the explanation—not from a favorable-looking total alone.

Put the decision into a QuizFlow survey

In QuizFlow Labs, a Multiple Choice block can hold the direct question and its labeled answer options. Use Single select when each respondent should choose one position on the scale. Check the selection mode deliberately so the person cannot select two incompatible ratings.

Start with one consequential decision, one specific question and an answer set that preserves the distinction you need. Preview it as both an experienced customer and someone who cannot judge yet. Review what each answer would lead you to investigate before adding more questions.

QuizFlow Labs helps businesses build interactive decision flows that turn answers into useful next steps. A well-chosen survey question supplies the evidence for that work; the software cannot decide what a vague rating meant.