Testing Times - How Can Psychometric Testing Be More Effective?
29-Jun-2011By Carolyn Boyd
Every week, hundreds of staff at Easternwell fly in and out of mine and gas sites. Much of their work maintaining and operating drilling and well servicing rigs is heavy, and potentially dangerous. “For us safety is a critical part of our business,” says Kyle Koziol, Easternwell's executive general manager of HR. Since December, Easternwell, whose clients include mining behemoths BHP and Rio Tinto, and gas giants Santos, QGC and Chevron, has been using psychometric tests to evaluate candidates for jobs in mining, oil and gas services.
The company is part of a growing number of businesses that are turning to psychometric tests to inform their recruitment now that such assessments are available online. Some estimates say that up to 70 per cent of blue chip businesses use psychometric questionnaires, and smaller companies are also showing an interest.
About a third of the candidates lining up for a job with Easternwell sit a psychometric test, and Koziol expects that to grow.
“We currently use assessments during recruitment in one part of our company and we are looking to expand on that,” Koziol says. “We started with a safety assessment which is an area that is critical for our business and we are looking to build on that with some of the other assessments that are available.”
Psychometric testing has its roots in the Chinese civil service in 1000BC, when a standardised test was used to assess job applicants, says Ian Florance, secretary of the European Test Publishers Group and MD of OnlyConnect. It entered the business realm more than 60 years ago, during the Second World War. “The American armed forces introduced it because they were very worried about the relative intelligence of people in the armed forces,” explains Florance.
A GROWTH INDUSTRY
Cherie Curtis, the head of psychology at one of Australia's half-a-dozen providers, Onetest, says that a decade ago assessment companies were educating businesses about the ins and outs of psychometric testing. Now, though, clients have a sophisticated knowledge of the process.“Ten years ago the market was using it in a considered way so [it was only for] certain roles and certain organisations,” says Curtis. “Assessments usually took a fair bit of time, and at that early stage they were probably more expensive.” But from that point to “where we stand now, the uptake was fairly rapid”.
Psychometrics are used across all industries. “All sectors use it, government and private, ranging from small to large companies,” says Draga Jevtic, a consultant organisational psychologist at Pearson Talent Assessment.
FLAWS IN THE SYSTEMS
While not denying psychometric testing can offer valuable insight, Florance worries it misses the big picture that could be really valuable for businesses. “The tests are really about how an individual differs from another person,” he says. “They don't take into account, or they only do, in a very minor way, how the social scene, the environment, the people you're with, and the context that you operate within, affects what you do.”
Florance has been closely watching the research of Human Insight, a British consultancy that specialises in “business ecology”. “[They] look at systems in an organisation and see how people are affected by the systems around them and the other people around them,” he says. “They map the whole organisation rather than just look at the individual people and add them together. If you just add them together, you're missing out on the total effect of the whole system and what the environment is actually doing to you.”
However, Curtis refutes the idea that assessments just look at the individual and the job they are applying for, without taking into account a broader perspective of the organisation. Onetest carries out a values audit across its client's business. Coupled with individual values assessments of candidates, it provides a picture of the match between the candidates' personal values and the culture of the organisation.
“We're actually asking a series of questions across the organisation and generally using that profile, which shows the organisations what that profile is versus what the perception is,” says Curtis. “That information can be very useful not only in that recruitment step, but also in terms of understanding differences in different pockets of a business within the culture. Culture can be very distinctly different for different people in different locations. And having an organisation gain that awareness is powerful.”
Ian Florance identifies another possible flaw in testing. As the West moves from a manufacturing-based environment where traditional male skills were sought-after, to more services-related industries with greater female participation and softer skills coming to the fore, Florance wonders whether the tests have kept up.
There's another grey area, as well. The question of whether psychometric tests can deal with different work styles as new generations hit the workforce. For example, younger people are more used to communicating using Facebook, Twitter and texts. They are bombarded with thousands of pieces of information. They don't so much need to know how to store it, rather how to find it later.
And as the nature of businesses continue to evolve, driven in some cases by the emerging trend to collaborate with customers online when developing products and services, the skills that are needed strongly lean towards questioning and listening, and the ability to touch multiple points on a global or at least, regional, scale.
Despite his questioning of the gaps in psychometric testing, Florance says he's still very much in favour of their use, particularly in recruitment where “traditional interviews are about as predictive of work success as flipping a coin”. “We're hugely influenced by irrelevant factors, we're subconsciously influenced by what someone looks like, what they say,” Florance says. “On the whole at an unstructured interview you will have made up your mind about whether you're going to employ this person within about the first minute.
“In recruitment what tests do is allow you to assess your unconscious prejudices, challenge them with objective evidence and structure an interview using questions generated from the assessment. Structured interviews that you spend some time preparing and use tests with, are extremely predictive of work success.” Jevtic says testing cuts through the spin that candidates use when seeking a job. “We all choose referees who are going to say positive things about us,” she says. “Assessments provide an objective measure.”
So does psychometric testing really work? The question draws a blunt response from business consultant and former president of the Australian Psychological Society Bruce Crowe. “The testing wouldn't be used by so many people if in fact it wasn't delivering some outcome, because business as you know, is very results-orientated.”
Crowe says the nature of business may have changed in recent years but so too have the tests. “We've had 50 or 60 years of modern test development, and we've got to a stage where with computerised testing you can tailor a test by picking and choosing items that go into it,” he says. While it's true that testing can be tweaked to move with changes in the skills companies look for or the norms emerging in test results, whole new assessments take years to get right.
“The problem with development of good assessments is that they take five to 10 years to develop to ensure good reliability (consistency of results over time), and validity (that they are measuring what they were developed to measure rather than other variables),” says Jevtic.
Nevertheless, adapting existing tests is useful, and is a lot easier to do now that testing has migrated online. Take the skill of critical thinking, which is now high on the radars of employers and testing firms. “It has been identified as the number one quality most lacking in our next generation of leaders,” says Jevtic. “It's also been identified as a top business skill over the next five years, and high for on-the-job success. [US President Barack] Obama identifies it as a skill of this century.”
Another element of psychometric testing that needs regular review, says Jevtic, is the norms that individuals are compared to. “When we develop norms, it's hundreds of people, they're not small sample sizes,” she says. “We have tested people globally so we can say „this is an accurate reflection.”
But online testing could allow companies to update these norms on a minute-by-minute basis, being informed by the results of each and every new candidate. It‟s happening but not across the board – yet.
“It depends on the particular platform that the various publishers use,” explains Jevtic. “Some have this capability while others are updated from the back-end on a regular basis. Either way the norms will be more up-to-date than when they were paper-and-pencil and often not updated for a number of years.”
WHAT HR MANAGERS NEED TO KNOW
Tests are one thing, but how they are used can be what really matters. “The HR manager either needs to train themselves up in 'how do I interpret psychometric assessment properly', or outsource that to someone who knows how to interpret it properly,” advises Jevtic. “A lot of assessments today give you a computer-generated report, but that's insufficient because … the computer doesn't know the person, the computer doesn't know the position.”
At Onetest Pearson, training is a mandatory requirement for all clients using assessments to provide the necessary insight into how the assessments should be used.
“The assessment should never be used in isolation, it should always go back to the questions, what are we trying to aim for here? What are we trying to ascertain?” says Jevtic. “In the selection process it is about always matching the outcome of the selection to the key selection criteria.” Reliable and balanced assessments can be used incorrectly but that doesn't mean the assessment is a bad assessment, it just means the user isn't using it the way it needs to be used. HRm
Comment