Using reliable research results in order to formulate and choose employee assessment methods is known as evidence-based management – something every leader should strive for.
Looking at the research on executive employee assessment effectiveness offers some fresh perspectives on what works — and what doesn’t.
In this post I’m going to employ this approach to show you what available research had to offer on employee talent assessment for hiring.
Specifically, I wanted to know what criteria are most effective in the selection of talent, and why some methods work while others fail.
In any area, knowing what works provides you with the confidence that the tools you choose to utilize are based on proven, predictable results.
Just as the stock market ebbs and flows, the “talent wars” never end.
Knowing how to pick the best people is the fuel that makes a business run smoothly and productively.
Remember: Numbers don’t get results — people do!
So You Think You Can Dance?
Do you know the most effective employee assessment criteria for selecting top talent?
We all know which methods are most prevalent and almost universally used, and we assume they work because everybody relies on them.
Question: Which of the following are generally accepted to be the most accurately predictive selection criteria of successful job performance?
A) Years of Job Experience
B) Years of Education
C) Assessment Centers
Answer: Actually, none of these are in the Top 5 list of predictors!
In my investigation I found a very interesting paper on this topic published by Frank Schmidt and John Hunter in the Psychological Bulletin in 1998.*
These two researchers analyzed the pattern of relationships observed in peer-reviewed journals over a span of 85 years to identify which employee assessment methods were best and worst as predictors of job performance.
They used a method called meta-analysis to do this. This method essentially reveals the overall patterns indicated by the weight of evidence rather than the particular quirks of any single study.
*The Validity & Utility of Selection Methods in Personnel Psychology: Practical & Theoretical Implications of 85 Years of Research Findings.
So What’s the Bottom Line?
The research conducted by Schmidt and Hunter demonstrated that work sample tests (tests that evaluate whether people can actually do key elements of a job; i.e. can a secretary type or a programmer write code), general mental ability assessments (the GMA is an abilities test for critical thinking skills, verbal reasoning, mental alertness, etc.), and structured interviews (such as web-based assessments) had the highest validity of all methods examined.
Interesting Results
This research study found three combinations of methods that were consistently the most powerful predictors of job performance. These most effective assessment solutions were:
- GMA plus a work sample test
- GMA plus an integrity test
- GMA plus a structured interview
Measuring Job Performance:Results of findings by Schmidt and Hunter | ||
Combined Methodologies: | Description: | Validity Coefficient‡ |
GMA† plus a work sample test |
Hiring someone and then evaluating whether they can do the work |
.63 |
GMA† plus an integrity test | Assessment of honesty | .65 |
GMA† plus a structured interview |
Unstructured interviews are generally weaker indicators |
.63 |
† General Mental Ability (GMA) combined with other assessment methods
‡Validity coefficient: A measurement of how strongly the individual method is linked to performance.
Key Takeaways
These findings about combined methodologies are far more relevant to executive assessment tools than evaluations of individual test methods as quantifiable indicators of which blend of methods works best.
As Schmidt and Hunter concluded, the obvious advantage of the latter two combinations is that they can be used for both entry-level employee selection, and assessment of experienced employees.
Here is the rank order of the 19 indicators examined, grouped by validity coefficient measurements to show how strongly each individual method is linked to performance:
Rankings of Individual Performance Indicators | |||
Ranking Indicator | Validity Coefficient | Notes: | |
1 | Work sample tests | .54 | |
2 | General Mental Abilities (GMA) tests | .51 | General mental ability |
3 | Employment interviews | .51 | Structured, as opposed to unstructured |
4 | Peer ratings | .49 | |
5 | Job knowledge tests | .48 | Test to assess how much employees know about specific aspects of the job. |
6 | T & E Behavioral Consistency Method | .45 | On the premise that past behavior is a reliable predictor of future behavior. In reality, this method describes success gained through work, training and other related experience while aligning accomplishments with competencies required on the job. |
7 | Job tryout procedure | .44 | Employees go through a trial period of doing the entire job. |
8 | Integrity tests | .41 | Designed to assess honesty |
9 | Employment interviews | .38 | Unstructured as opposed to structured |
10 | Assessment centers | .37 | |
11 | Biographical data measures | .35 | |
12 | Conscientiousness tests | .31 | Essentially, do people follow through on their promises, do what they say, and work doggedly and reliably to finish their work? |
13 | Reference checks | .26 | |
14 | Job experience | .18 | Measured in years |
15 | T & E point method | .11 | |
16 | Education | .10 | Measured in years |
17 | Interests | .10 | |
18 | Graphology | .02 | Handwriting analysis |
19 | Age | -.01 |
Points to Think and Dialog About
From my point of view I find these indicators to be most interesting:
- Years of Education
- Years of Experience
- Age
Research clearly demonstrates that these indicators don’t seem to matter much, yet they are ever-present in selection requirements.
You can present the following questions to engage your management in an active dialog about executive evaluation practices. Doing so will serve to make you and your organization more aware of how your evaluation process really works, and why you should or should not continue to hire by the criteria you are using now.
Points for Consideration:
- Why is a college degree used as a screening requirement for selection and/or promotion?
- Why does the executive search community continue to be reluctant to present executives over 50 years of age to their clients?
- Why is “years of experience” almost always considered first in terms of employment requirements, and utilized first in resume screening by recruiters and hiring managers?
- Why would any “best-in-class” organization use these apparently sub-par predictors as their primary screening criteria?
Conclusion
For these reasons and many others, the online employee assessments we offer at rd&partners ensure that you measure potential in the right areas instead of areas that really don’t correlate to successful hire at all.
How would you evaluate the success rate of your current executive assessment and employee assessment program?
What are your thoughts on the findings of this research, how would you address the above questions, and what would you add to this list? I’m eager to hear your take on the vital topic of talent assessment.
Leave me a comment below, and let me know what you think and how your organization is dealing with these dilemmas.