ReliableSurveys.com
Home  |  Reliability  |  Validity  |  A Technical Discussion  |  Different Methods = Different Results 


What's the Scaled Comparison

The Scaled Comparison® is a proprietary assessment methodology that improves dramatically the accuracy of surveys and evaluations.

  1. You know the traditional survey.  They give you a statement about something.  You get to say you agree or disagree, or maybe you get to say how much you agree or disagree with the statement.
  2. Then there is the evaluation — like evaluating people or evaluating how you were treated as a customer.  You can check one of 5 boxes from low to high.  Maybe there are 7 boxes or 10, but the process is the same.
  3. Finally, there's multiple choice.  You are asked a question, and given 4 or 5 alternatives. You answer by selecting one of the alternatives.

That's about it.  Almost all of the ways we have of finding out how people think about something or someone fall into one of those three methods.
The Scaled Comparison is different.  It doesn't ask you if you agree or disagree.  It doesn't ask you to "pick a number between 1 and 10".  Nor do you have multiple choices.


The Scaled Comparison is different.  It doesn't ask you if you agree or disagree.  It doesn't ask you to "pick a number between 1 and 10".  Nor do you have multiple choices.  It asks you compare 2 issues, or 2 persons and indicate which you think has more of something — more importance, higher performance, describes better — than the other.  You may like both of the "things" or neither of them.  But you are forced to choose.  That's why it is called a "forced choice" approach.

Some questionnaires out there use a simple version of the forced choice method.  It's called the "paired comparison" and it works very well for certain situations.  You get to choose simply one or the other of two statements, even if you think they are both equal, or both the same.

The Scaled Comparison goes a step further, by giving you a way of saying the two things are equal, or slightly different or much different.

In case you're thinking that any one of these methods is just as good as another, we urge you to see Different Methods, Different Results for an example of how a comparative method can yield very different results than voting or scoring techniques. 


Online QuestionnairesThis tool is available online and completely customizable.

The Scaled Comparison is a tool for prioritizing.  That means it can tell you the highest performers and the lowest performers in a group of people.  Or what are the most important and least important skills to train.  Or the best and the worst actions to take. Here are some of the applications...

Performance Appraisal Using the Scaled Comparison

The Scaled Comparison can be used in two ways to evaluate performance.
  • Performance Ranking
  • Performance Profiling

Performance Ranking

Performance Ranking will be the method of choice if the organization needs to make decisions that are comparative, such as for merit pay, promotions or downsizing.  In these situations, it is necessary to determine relative performance in a manner that is valid and defensible.  While it is not recommended for performance feedback or development, the process assists organizations who need to know who the highest performers and the lowest performers are.

To assess peoples' relative performance, they are compared using a scale, like this:

Question: Which person demonstrates greater Leadership ability?
Doug Reichman Jim Craig

People being evaluated like this need to be compared with others many times, not just once.  And valid evaluations of job performance need to consider performance on multiple criteria, not just a single overall measure.  So the actual process looks more like this:

Question: Which person demonstrates greater Leadership ability?
Doug Reichman Jim Craig
Rene Bouchard Sara Salazar
Jim Craig Rene Bouchard
Sara Salazar Doug Reichman
Clancy Broom Sara Salazar
Doug Reichman Clancy Broom
Clancy Broom Rene Bouchard
Sara Salazar Jim Craig

This form of performance appraisal:

  • Is used to build consensus about relative performance of organizational members.
  • Evaluates people in their natural job families.
  • Can evaluate a few or thousands of people in the same job family.
  • Uses multiple evaluators.
  • Can use Superiors, Peers, or Subordinates as evaluators.
  • Uses multiple evaluation criteria.
  • Reports on evaluator behavior as well as the people being evaluated.

Performance Profiling

This is a second way to evaluate performance, but with the objective of determining their "fit" or suitability for the known skill requirements of their job.  You would choose this approach if you wanted to know the "gaps" between a person's actual performance and the ideal required for their job, or for a job you have in mind for them (see Performance Profiling or Development Profiling or Strategic Skill Profiling).  The results are ideal for performance feedback, assessing potential or for designing career development plans for a person.

The following format is used to compare various characteristics of the person (or position) being studied:

Question: Which statement is a better example of how Clancy Broom ACTUALLY conducts himself as Supervisor?
Doesn't give people more than they can do Decisions made at the lowest possible level

These statements can be observable behaviors, skills and abilities, or personal characteristics.  The illustration that follows shows how a questionnaire might appear.

Question: Which statement is a better example of how Clancy Broom ACTUALLY conducts himself as Supervisor?

This form of evaluation:

  • Evaluates behaviors, skills, and characteristics
  • Uses an Ideal profile defined internally by the organization.
  • Employs input from 5 to 10 evaluators for each person profiled.
  • Provides complete protection against errors in the results due to bias, intentional manipulation, or misunderstandings.
  • Is cost-effective and time-efficient. The entire profiling process may be designed and conducted by in-house personnel.  Little or no consultant help is required.
  • Works well with just a handful of key people, or your whole organization.

Scaled Comparison Surveys...

are used to build consensus about organizational issues, such as Customer Service Assessment, Team Effectiveness, Readiness for Total Quality / Six Sigma, Training Needs Assessment, Values & Culture Analysis, Human Resource Alignment.

Question: In an ideal situation, which do you believe is more important to achieve a highly effective organization?

This form of assessment:

  • Obtains the consensus quickly, efficiently, and confidentially.
  • Can study as many as 50 to 60 values, priorities or issues.
  • Uses multiple questions, i.e., Which is more important?  Which is being rewarded more?  What's more in need of attention?  What will be more important to success in the future?
  • Will deliver valid data with groups as small as 5 people; has been used to assess thousands.
  • Can report differences between the views of subgroups of the respondents.

Click Here for a more technical discussion of the Scaled Comparison.

How do I get more information from ReliableSurveys.com?




 posted: 19:07 - 06.08.08 [an error occurred while processing this directive]