What questions go in a message testing survey?
TL;DR
- Ask clarity, relevance, believability, differentiation and action for each message, then a forced choice across finalists.
- Start with an open "what is this saying?" question before any scales, so people's own reading comes first.
- Wording moves answers: in a classic 1940 split-ballot, 75% would "not allow" a kind of public speech but only 54% would "forbid" it.
- Neon Blue fields this questionnaire on a simulated audience of your buyers by segment, labeled as simulated. Run your first study free
Neon Blue builds simulations of human behavior. A message testing survey asks people to read one message and answer five kinds of questions: what it means to them (clarity), whether it matters to them (relevance), whether they believe it (believability), whether it sets the brand apart (differentiation), and what they would do next (action). It then asks them to compare the finalists head to head. Each person should see one message first, so the first message doesn't color the next. The wording of each question matters as much as the message, because small word changes move answers.
What is the full message testing questionnaire?
Show one message per respondent, then ask in this order:
- In your own words, what is this message telling you? [Open]
- How clear is this message? [1 Very unclear to 5 Very clear]
- How relevant is this to you personally? [1 Not at all to 5 Extremely]
- How believable is this message? [1 Not at all to 5 Completely]
- How different is this from what other brands in [category] say? [1 Exactly the same to 5 Completely different]
- After reading this, how interested are you in [product]? [1 Not at all to 5 Extremely]
- What, if anything, would you do next? [Look it up, visit the site, ask someone, nothing]
- Which word or phrase stood out most, and why? [Open]
- Is there anything in this message you don't believe or don't like? [Open]
Then show all finalists together:
- Which of these messages would make you most interested in [product]? And which least?
Questions 1, 8 and 9 are the ones teams skip to save time, and they explain the scores. How a best-worst ranking works, and when to use it, is in MaxDiff or conjoint analysis.
Why does question wording matter so much?
In 1941, Donald Rugg reported a split-ballot experiment in Public Opinion Quarterly. Half a national sample was asked whether the United States should "allow" public speeches against democracy; the other half whether it should "forbid" them. The two questions mean the same thing. Yet, as Howard Schuman and Stanley Presser later laid out in Questions and Answers in Attitude Surveys (1981), 75% said "not allow" while only 54% said "forbid". Their own 1974 replication found a gap in the same direction: 44% "not allow" against 28% "forbid".
For message tests, the lesson is to keep wording identical across messages, avoid loaded verbs in the questions, and never word a scale so one end sounds harsher than the other. The same kinds of wording traps in concept work are covered in concept test questions.
How do you read the answers?
| Measure | Strong message | Warning sign |
|---|---|---|
| Open meaning | Most answers match the intended idea | Answers scatter or name a different benefit |
| Clarity | High scores and matching open answers | High score, but open answers show a misread |
| Relevance | Strong in the target segment | Strong only among people outside the target |
| Believability | High, with few objections in question 9 | Many people asking for proof |
| Differentiation | Rated different from rivals | "Everyone says that" in open answers |
| Action | A real share would do something | "Nothing" leads even when scores are good |
A message can score well on clarity and fail on differentiation; that is a category-generic message and rarely wins share. When believability is the gap, the fix is often a proof point, tested through claims testing.
What does a message test look like on a simulated audience?
Hypothetical: a meal-kit brand has four headline messages for a winter campaign and wants one to lead in ads and on the landing page.
- Audience: US adults who cook dinner at home at least four nights a week, split by current subscribers, lapsed subscribers and people who have never subscribed, and by household size.
- Messages: "Dinner decided in two minutes", "Restaurant meals at home for less", "Fresh ingredients, zero waste", "Cook something new every week".
- Design: one message per respondent with questions 1 to 9, then the forced choice, on a simulated audience of home cooks.
Example output layout (illustrative, not a real result)
| Message | Matches intended meaning | Clear (top-two) | Relevant (top-two) | Believable (top-two) | Most-picked share | Main objection (coded) |
|---|---|---|---|---|---|---|
| Dinner decided in two minutes | __% | __% | __% | __% | __% | Coded theme |
| Restaurant meals for less | __% | __% | __% | __% | __% | Coded theme |
| Zero waste | __% | __% | __% | __% | __% | Coded theme |
| Something new every week | __% | __% | __% | __% | __% | Coded theme |
If lapsed subscribers pick "restaurant meals for less" but flag price as their main objection, the message is pulling on the very doubt that made them leave. A different message might win them back, which is a positioning question as much as a copy one, and the growth team can test the winner on the landing page.
What should a message test not do?
- Don't test a message and a design at once. Put messages in plain text first. Creative execution is a separate screen; see testing ad creatives before media spend.
- Don't skip the target cut. A message that wins overall and loses in the target segment is the wrong winner. Build cuts as in finding your target audience.
- Don't stop at the simulated read for a big campaign. Take the top two to a short live check with real buyers before the media plan locks.
Questions about message testing surveys
How many messages should you test at once?
Four to eight in one study works well. Show each person one message for the full questions and all finalists for the forced choice.
What scale should a message testing survey use?
Five-point labeled scales are standard. Keep the same scale for every measure so results can be compared across messages.
Should message testing be monadic or comparative?
Both. Monadic first for clean scores on each message, comparative last to see which one wins when people choose.
How does Neon Blue run a message testing survey?
Neon Blue fields this questionnaire on a simulated audience of your buyers, codes the open answers, reports every measure by segment and labels results as simulated. Start a free first study.