 |
Why Do Our Questionnaires Look Different?
Because we use a methodology superior to traditional surveys.
First, an important distinction. All questionnaires ask for information, some of which is objective, or factual, such as, "How many children do you have?" and some is subjective, such as, "How much do you like your children?" The focus of our discussion will be on the methods of asking subjective questions.
As the word "subjective" implies, there is a judgment involved, which must first be made, then reported. The way a question is presented affects the judgment that will be made, which, in turn, determines the kind of data we will get. First, we will look at the most common approach to asking subjective questions.
Rating Scales — Absolute Questions
These are the questions we've all seen before. In fact, they are so ubiquitous, most people think there's only one way to ask questions about opinions and attitudes. We're talking about the rating scale method. Here's what a rating scale looks like:
Why Would I Want to Ask Questions This Way?
There's a practical answer and a scientific answer.
Practically Speaking...
The decisions we make in life for which we need good judgments are almost always comparative, rather than absolute. We have to make choices about how we will spend our time, allocate our resources, share our knowledge, choose right from wrong, choose right from right — these are all comparative choices. They aren't necessarily mutually exclusive, but they do involve priority decisions.
In the workplace, almost all decisions are comparative, because almost all resources — space, financial, time, human, opportunities — are limited, rather than unlimited. If one person is promoted, others are not. If some projects are approved, others are not. It is a constant process of deciding which issue is more critical to the objectives we are moving toward. We may think we are focusing on "how important" something is, when we are really looking at "which is more important — A or B?" Rarely do we focus on the absolute importance or value of something in making decisions.
Even when an organization uses an absolute, rating scale method to assess something like employee satisfaction, they don't look at the results for absolute information. They immediately start asking questions like "What are the salient issues, what needs immediate attention?" They want to know the important concerns of the employees, or perhaps the things the organization is doing well. They don't look at a single number and say, "Hmm, benefits are 63% — that's interesting." Any single absolute result makes sense only when compared with other issues.
Scientifically...
People are better — exponentially better — at comparative judgments than absolute. As children, we learn to compare the relative weight of two objects in our hands. We know how to identify which of two persons is taller years before we learn about tape measures and average height.
In laboratory testing, individuals told to judge the length of a black line painted horizontally on the wall 10 feet away could rarely guess within 4 or 5 inches of the actual length of the line. But when presented with two lines, and asked to guess how much longer one line was than the other, they were regularly able to guess within 1/4 to 1/2 inch of the actual difference. The same studies have been repeated with different objects, distances, and sensory modalities with highly consistent findings. We are simply better at comparative judgments than absolute.
Conclusion
If your goal is to gather reliable data about subjective issues, employ a comparative questionnaire approach rather than one that uses absolute, or rating scale questions. Comparative assessment tools are available here. We have been using them since before the internet, and we are experts in measurement theory. Contact us for more information on how we can help you.
|
|
|
