Too Fancy? Thoughts on Weighting Schemes for Pay Survey Data

Whether we are using sophisticated software products or basic Excel spreadsheets to organize and analyze compensation survey data, we compensation professionals have a tendency toward using elaborate weighting schemes in developing overall market values for jobs.

What do I mean by this?  Well, let's say you are "market pricing" (gathering competitive market pay data on) a Software Development job.  Let's further say that you've identified three good "matches" from three separate salary surveys for this job, as noted below:

Survey 1:  Job title - Software Developer

Survey 2:  Job title - Software Developer - Intermediate

Survey 3:  Job title - Software Engineer 2

A simple approach would be to weight each of these survey jobs equally - or, essentially, calculate a simple mean or average of their values.  A more complicated approach would involve developing a "composite" (a weighted average of sorts) which reflects weighting each survey job differently.  For example:

Survey 1:  Job title - Software Developer (10%)

Survey 2:  Job title - Software Developer - Intermediate (60%)

Survey 3:  Job title - Software Engineer 2 (30%)

Why, or on what basis, would we treat these survey jobs differently?  Reasons abound and can include (but are not limited to):

1. A desire to reflect the fact that some survey matches are a better fit than others
2. A desire to reflect the fact that some survey sources are a better fit than others
3. A desire to place more or less emphasis on a particular industry sector or geographic area
4. A desire to weight each survey job proportionate to the number of employees it represents (e.g., a piece of data that reflects the pay rates of 400 employees would be weighted twice as much as one that reflects the pay rates of 200 employees) or, in other words, to develop an "employee-weighted" average.

My take?  I have to say that I lean toward simplicity, and advice my clients accordingly.  The simpler the survey weighting scheme, the better - unless there is a logical and compelling reason to get "fancy" (and sometimes there truly is).  In my experience, however, getting fancy often becomes a slippery slope to a place that is difficult to explain and defend.  Not a good place to be in these times of greater pressure for pay program transparency.

P.S. If you must get fancy, make sure and clearly footnote your weighting approach and rationale.  You - and anyone who must follow behind you and unravel your logic - will be glad you did.