Stemler’s Study Reveals that Knowing What Not to Do Is a Critical Job Skill

Situational judgment tests (SJTs) have become an increasingly important tool for predicting employee performance.

In a recent study, Steven Stemler, associate professor of psychology, and two executives at pre-hire assessment firm Aspiring Minds asked current employees at several firms in India to review scenarios and then pick the “best” and “worst” choices from a set of options.

The colleagues found a statistically significant correlation between job success and those who correctly identified the ‘worst’ answers to scenarios.

Their results were surprising.

“What we found in our research is that the ability to correctly identify the ‘worst’ response to a situation is a systematically different skill than the ability to identify the ‘best’ response, and the two may not even be related,” Stemler said.

As a result, Stemler, Varun Aggarwal and Siddharth Nithyan produced a paper titled “Knowing What Not to Do Is a Critical Job Skill: Evidence from 10 Different Scoring Methods,” which was published in the September 2016 International Journal of Selection and Assessment.

Consider a student trying to decide how best to prepare for an upcoming exam, business leaders trying to decide on a corporate strategy moving forward, or citizens trying to decide who to elect to public office.

“The first thing people tend to do in all of these situations is think about their ideal solution. But what happens when that option is not realistic or just won’t fly due to certain constraints? Then you have to start evaluating all the alternatives,” Stemler said. “And from there, the thing that matters most changes a little bit. Though the other options may be less than desirable, there is typically going to be one ‘worst’ option that could lead to utter disaster.

The key finding from their research is that this ability of an individual to successfully identify the “worst” option (and presumably avoid it) is actually significantly more predictive of their future job success, as measured by both objective and subjective indicators, than is their ability to identify the theoretical “best” option.

“A critic might respond, ‘well, nobody would actually pick the worst option,’ but in fact, our empirical work shows that people do,” Stemler said. “Indeed, if everyone was able to correctly identify the “worst” option, the data would not be a significant predictor. So the very first point to understand is that even though most people agree on the “worst” option to any given situation, there are still many people who will select that option because they don’t see it. And our new SJTs are rather good at identifying those people quantitatively.”

As a result of the publication, Stemler was featured in the Sept. 30 edition of The Wall Street Journal in an article titled “Hiring Tip: Find the Person Who Knows What Not to Do.”

According to Stemler’s study, hiring the best person for a job might be the one who can identify the worst-case scenario. That talent is evidence of “practical intelligence,” which includes knowing how to avoid pitfalls and potential crises. The findings, according to the article, suggest that managers can improve their odds of hiring good performers by 10 percent if they start selecting for the ability to recognize poor options rather than the so-called best ones.