Taking a Stand for Quality in the Pay Data Battle

Today, any motivated employee can gain access to salary  data which suggests, nay proves, that he or she is underpaid by many thousands of dollars.  Yes, even in these tough economic times, and you know the practice will only increase as the economy recovers. 

One of the most frequent questions I get from my own clients is how to deal with the line of employees out their door waiting, Internet pay survey printout in hand, to discuss their compensation.

I recommend meeting this challenge head on by focusing on pay data quality and establishing a policy that features specific quality criteria that all pay data must meet (whether brought to the table by Human Resources or an employee) before it will be considered relative to the organization's compensation practices.  Armed with a formal policy on pay data quality, the HR pro is positioned to  have a productive discussion about the specific data an employee brings in.

Let's face it.  Data of questionable quality exists both on the Internet and in traditional hard-copy format from publishers who should know better.  The good news, though, is that the same quality standards should apply regardless of the source of the pay information.  Here are some of the standards I recommend for assessing pay survey quality, presented for your consideration in developing your own policy.

Survey details its methodology.  The survey data provider should detail its process for contacting or soliciting participants, collecting and analyzing data, and checking data quality and validity.  A lack of transparency around survey process and method only makes me wonder what they are trying to hide.

Survey data is collected from an independent, verifiable source.  I would consider the Human Resources, Finance or Payroll departments of an employer to be independent, verifiable sources of pay data.  Surveys based on data submitted by recruiting firms or self-reported by individual employees (both of which stand to gain by inflating pay rates) are not considered reliable by most compensation professionals.

Survey identifies participants.  If not a list naming participating organizations, there should at least be a demographic profile which outlines information like their number, size, location, industry and other characteristics.  If you don't know where and from whom the data is drawn, how can you determine whether it truly represents your organization's labor market? 

Survey provides job descriptions adequate for matching.  Proper job matching is the foundation for successful market pricing.  Matching by job title alone is fraught with peril given the wide variances in titling practices that exist among different organizations and industries.  Survey job descriptions should include education and experience specifications, as well as a listing of job purpose and typical responsibilities. (And it is sometimes necessary to remind the employee that the employer sets job price by the job responsibilities and requirements, not the years of experience they happen to possess.)

Survey reveals effective date of data.  Its important to know how current the data is so that its validity can be ascertained.  Given the turbulence in today's labor market, data that is too old -- regardless of whether or not an aging factor has been applied to bring it forward -- may no longer reflect the realities of a particular job's competitive value.

Survey notes sample sizes.  How many organizations and employees are represented in a particular piece of data?  Obviously, data based on 200 employees is more meaningful than data based on 5 employees; without this information it is impossible to determine whether a markedly high or low pay figure is valid, or simply the skewed result of a small sample size.

With formal pay data quality policy in hand, my advice to the HR pro who is approached by an employee with their own competitive information is as follows:

1. Thank them for their efforts in bringing the data forward. (Yes, I know, but really.  If it turns out that the data is valid, they may be doing you a favor.  Just keep your mind open to the possibility.)

2.  Explain (and provide a hard copy of) the organization's policy on pay survey data and quality criteria.

3.  Sit down with the employee and review their information together to ascertain whether it meets the organization's criteria.

To the extent that the employee's information meets the quality criteria (and it represents a valid match for his/her job), it ought to be taken seriously and considered in light of current pay practices for the position and the particular employee.  To the extent that it does not, the occasion should provide a teaching moment for the employee (although the astute HR professional may wish to go afterwards and double check the data used historically to set pay practices for the position, just to ensure that the organization is standing on firm ground).

What's your practice and approach for responding to employee data and the quality questions it often raises?  Got any tips to share with us here?

Image: Creative Commons Photo "Terra Firma" by carbonNYC

Comments

[Amy]

I'm sure many of us read this with our hands waving in the air. Yes! Yes! Other data elements we find important are how the survey cuts the data in terms of regional scope and the regression analysis methodology. We look for specific regions that match our labor pool criteria by job (e.g. managers = national, nurses = within the state, clerks = within the region) and we tailor certain jobs to our revenue/beds using regression. Too often we're approached with a magazine article that uses national data skewed by the small number of participants, heavy on the coastal regions that inflate the numbers significantly. Bravo for another great observation!

[Blair Johanson]

Ann,

Excellent comments and recommendations on the pay data quality battle. We have found certain well-known internet based pay data providers to be 20%+ above market pay means. It is prudent and efficient to have a market pay study policy process. Quality market salary data is becoming harder to obtain due to rising costs and an unwillingness by the HR Pros to complete numereous requests for internal pay means and range information. Job matching is critical if the organization seeking market pay data does not have an internal job valuing process. We really encourage our clients to utilize the combination of an internal job valuing process and a healthy amount of market pay comparisons as possible.
Blair

[Ann Bares]

Amy:

I appreciate the enthusiasm - glad you found the post "on message"!

And thanks for the added emphasis on matching up to your organization's defined labor pool. Not enough to just have quality survey data, it must be quality survey data that captures the right labor market.

Blair:

You bring up an important point going forward - quality market survey data is becoming harder and more expensive to obtain, just when (and perhaps in some part because) the market is becoming more and more dynamic and fast changing. This will have interesting implications going forward, for sure ... but that's the topic of another post...

Thanks - both - for sharing your thoughts and comments!

[Roland]

Why not counter the employees data with your own valid data?

You suggest, "Sit down with the employee and review their information together to ascertain whether it meets the organization's criteria." Nowhere do you suggest being sure your own data are valid or criteria anything but arbitrary.

[E James (Jim) Brennan]

When presented with the typical outlandish number found on the internet ("so it MUST be true!") with mysterious or self-nominated sources, refer them to http://bls.gov/bls/blswage.htm where the Bureau of Labor Statistics hosts its free OES and NCS survey data. Updated versions of government data covering thousands of jobs titles are also available in the free online calculators at http://SalaryExpert.com which uses national statistical office data (BLS in US, StatsCanada, NatnlStatsOffice in UK, etc.) and presents the original source data. It is never as high as commercially-collected industry/size-specific data, but the authority of federal pay data cannot be contested.

[Ann Bares]

Roland:

Good point. I guess this post starts with the assumption that you have your own data act together - but if not, certainly that is the place to begin!

Jim:

Thanks for the tips and the links here - helpful!

[Benjamin Frost]

Hay Group has some more on this subject, through an article in Workspan last year - http://bit.ly/cGnKmv

For me, the "garbage in, garbage out" rule applies particularly well when looking at salary surveys - and that means it is well worth spending the extra time needed to get job matching or job mapping right.

[Ann Bares]

Thanks, Benjamin. This post wasn't meant to address the overall market pricing, survey quality process - a very important topic and deserving of its own separate treatment. This simply represents a few thoughts to help HR and compensation pros in dealing with employees who bring in their own independent pay data.

Certainly we - as the professionals charged with pay system design and management - have an obligation to ensure that we are building on a strong foundation of quality data. I appreciate the link to the well-written Hay article on the topic. I have also written a number of posts here addressing this, which readers might find helpful to revisit, including...

http://www.compensationforce.com/2010/03/dont-short-shrift-the-review-of-your-market-data-set.html

Thanks for adding to the discussion!