How to Make Good Surveys

An Underused Tool

Consultants should use surveys more often. They are cost-effective, seemingly impartial, easy to deploy and capable of producing data in the touchy-feely areas where evidence is hard to find, collect or quantify. Bain, PwC, BCG and Deloitte all publish survey-driven research, and even modest blogs use reader surveys to learn who their audiences are and what they want. Marketers figured this out ages ago. Instead of guessing what consumers want, ask them, and instead of asking one at a time, send a survey. Free tools have made the technique ubiquitous, and rightly so.1

A concrete example shows the payoff. In one reader survey, 121 people anonymously reported their savings rates: 55 percent saved a fifth of income or less, while 12 percent saved more than half. Data is a consultant's friend because it is apolitical and tests theories fairly, and surveys create data where none existed. Past applications include testing customer preferences in financial services, ranking software features, gathering feedback on presentation materials and even settling the venue for a holiday party.

Anchor Everything to the Objective

Good survey design begins before any question gets written. Clarify the research objective: what are you trying to discover, and do the questions and answer choices support it? An unclear problem statement guarantees a lost survey and wasted respondent time. Think the decision through in advance, asking what action you will take once the answer arrives, because a survey that changes no decision was never worth fielding. Then segment and target the audience so answers come from the right people, since mass-market blasts are dead and each survey should aim at a tribe. Getting personal has its place within limits, because segmenting responses requires some data on gender, age, affiliation, geography or income level.

Organize the Flow Psychologically

Question order shapes data quality. Put difficult questions in the middle, with easy questions at the beginning to warm respondents up and at the end when boredom and anxiety creep in. One thoughtful reader compared it to a difficult conversation with a stranger: you do not open with the hard questions, you build up slowly. Make everything easy to answer, favoring things respondents actually know and remember. And choose question types deliberately by working backwards from the analysis you intend, whether a table, a graphic or a cross-tabulation. If you need percentages, open-ended questions will betray you.

Write Simply and Fairly

The writing rules compound into credibility. Use short questions with simple words and no acronyms. Strip bias out of every question, because leading the witness toward your preferred answer is a cheap shot, unprofessional and sloppy work. Ask one question at a time and never bundle two into one. Make answer choices mutually exclusive and collectively exhaustive, removing all ambiguity about where a respondent belongs.2 Include not applicable as a choice, because forcing false answers corrupts the dataset. One experienced reader adds a subtle refinement: use an even number of answer choices to prevent respondents from straddling the middle. Finally, remember the four writing personas of madman, architect, carpenter and judge. Architect the question order, carpenter the wording and choices, then judge harshly and edit mercilessly until the survey earns its audience.

Think Like a Marketer

Design gets the questions right, and marketing gets them answered. Send the invitation with a clear title and an explicit call to action. Provide an incentive where appropriate, whether a gift card, a prize or simply a promise to share the results. Respect respondent privacy and say so, stating your policy plainly. Then split-test the invitation itself, pretesting two or three email subject lines to see which earns the higher click-through before committing the full distribution list. Response rate is a funnel, and every stage of it can be engineered.

Be Realistic

Sober expectations protect both quality and reputation. Respect readers' time by cutting unnecessary questions, and time the survey yourself, because anything over ten minutes is in trouble. Pretest on friends and colleagues rather than embarrassing yourself in front of thousands. And remember that survey takers are not average people. Email respondents skew computer-literate, agreeable and open to experience, with arguably lower opportunity cost of time, and honest analysis accounts for that lens rather than pretending the sample is the population.

From Responses to Decisions

Fielding the survey is the midpoint, not the finish. Clean the data first, removing straight-liners who answered every item identically and speeders who finished in implausible time, because both pollute averages. Cross-tabulate the key questions against your segmentation variables, since the interesting finding is rarely the overall percentage and usually the gap between groups. Report the base sizes alongside every chart so readers know whether a striking difference rests on 300 responses or 12. Then close the loop you promised, sharing headline results with respondents who were told they would see them. The final discipline mirrors the first: connect every finding back to the decision that motivated the survey, and flag honestly where the data came back inconclusive.

Get Started

Experience compounds faster than theory in survey work, so put more surveys into the field. Industrial platforms such as Qualtrics serve professional research programs, while free tiers of simpler tools handle most consulting needs with a ten-question limit that doubles as a useful discipline.3 The craft is part science and part art, and both halves improve only with repetition. The next unanswered question in your project is probably one well-designed survey away from becoming data.

• 1SurveyMonkey
• 2MECE principle
• 3Qualtrics