4 Reasons to Trust (valid and reliable) Psychometric Assessments

It goes without saying that human behaviour is incredibly complicated. It’s determined by an intricate combination of factors, and – as you can imagine – trying to predict how a person is going to behave, or react, or perform is no easy task.

Enter psychometrics, whose goal is to get accurate and unbiased insight into people’s mental abilities, personality, and behaviour. But how on earth is this possible?

1. There’s a lot of evidence that they work

Organisational Psychologists have spent over five decades researching, creating and rigorously testing psychometric assessments that are robust enough to predict when and why a given person will be successful or not in a given job. And as someone who is working towards becoming an Organisational Psychologist, let me tell you that these folks are an extremely hard to impress, detail-focused and highly sceptical bunch.

There’s now a large body of highly credible scientific evidence that demonstrates that a person’s results on a (valid and reliable) psychometric assessment can strongly predict a number of different work-related factors, including:

• Future job performance: how well they will learn new tasks, solve complex problems and perform on the job
• Organisational fit: whether they’re likely to share the organisation’s values and feel more committed and engaged in their job
• Safety behaviours: how likely they are to accept personal responsibility for safety at work and avoid risky behaviour
• Behaviour and personality: how someone naturally prefers to behave at work, the kinds of behaviours they have adopted, and how difficult it is to sustain behavioural changes
• Emotional intelligence: how well they can identify, understand, manage and use their own and other people’s emotions.

These kinds of assessments also give employers a standardised, fair and equitable way to compare candidates for a role, based on criteria that have been scientifically proven to predict success in a particular role.

2. They need to demonstrate reliability and validity

The next question has to be: how do we actually know that these assessments can really do what they say they will? It all comes down to two little words: reliability and validity.

These two properties are the foundation of psychometric assessments, and are the reason you can have confidence that psychometrics will help you identify and select the right people for a role.

So, what do we mean when we talk about reliability and validity? Let’s take a look at each concept on its own.

Reliability refers to the ability of an assessment tool to produce stable and consistent results. For example, a personality assessment should produce very similar results for the same candidate each time they complete it within a similar time period.

We can break reliability down a little further as well, into sub-categories that include test-retest reliability and internal consistency.

Test-retest reliability happens when we administer the same test to the same group of people several times, and achieve similar results each time. So, if someone is assessed as being a top performer in their first test sitting, a reliable test will give us a similarresult the second time they complete the same test.

Internal consistency reliability examines the consistency between the different items within a test. This means that if there are two or more items in an assessment that measure the same construct – for example, in a safety assessment, there might be multiple items that assess a person’s locus of control – we would expect that the same person will answer all of the items in a similar way.

Validity refers to the extent to which an assessment measures what it is intended to measure. For example, a measure of intelligence should measure intelligence, and not something else, such as memory. Like reliability, validity has a number of sub-categories which all need to be met for a test to be considereda legitimate psychometric measuring tool.

A particularly important sub-category is predictive validity. This concept is all about how well a test score can predict performance on a set of future criteria.

A nice example of predictive validity is the incredibly strong and rigorous scientific evidence that a person’s score on a cognitive ability test predicts their future performance at work. In other words it is very likely that the higher a candidate’s score on a (valid and reliable) cognitive ability assessment, the better their job performance will be.

There are many other cases of strong and rigorous associations between people’s scores on a particular construct and their subsequent performance at work, including:

• A robust association between a candidate’s score on a measure of work reliability or integrity and their rate of absenteeism from work
• A clear association between candidate’s score on a measure of safety and their likelihood of suffering a workplace injury or accident.

3. They go through an extremely stringent development process

Developing a psychometric test is not the kind of endeavour that can happen overnight. While anyone can pull together a quiz or questionnaire and deliver some results to people (certain magazines do this very well – and they’re fun to complete), constructing a proper, valid and reliable psychometric assessment is a whole other world of complexity.

Because they do have such stringent criteria to meet and need to prove that they can provide genuine information about a candidate’s suitability or ‘fit’ for a particular role, psychometric tests can take up to 10 years to develop.

To be taken seriously, the test developers have numerous hoops to jump through. One of these is making sure that the items in the test are measuring the construct they’re supposed to measure – and just that particular construct – as precisely as possible.

This involves conducting an intricate statistical analysis to determine which items should be eliminated from or retained in the item pool, and whether additional items need to be developed.

Yet another challenge is ensuring that psychometric assessments remain up to date and relevant. This usually means that tests need to be continually updated over time, based on feedback and new research in the field.

4. They have safeguards to prevent faking or response distortion

‘But wait!’ you may say. ‘This is all very well and good, but what about candidates giving the answers they think you want in an assessment?’ And that’s a really good question.

Obviously, when candidates are applying for a job, they’re motivated to show you their very best side. This also means that they’re likely to be tempted to give fake or distorted responses on an assessment, such as telling you they’re more reliable than they really are.

This is a question that psychologists have pondered for many years, and there’s a whole body of psychological literature dedicated to it. From all of this research, there are a number of different – and effective – ways we can reduce the opportunity for candidates to fake their responses, including:

• Verification testing: candidates complete the same assessment (with different questions) a second time under supervised conditions to verify their original results
• Validity scales: checks are built in to the assessments (by certain questions or algorithms) to detect whether candidates are trying to present an overly positive image of themselves or their behaviour

Making candidates aware of the consequences of faking: some psychometric assessment providers (Revelian included) also collect some fairly sophisticated forensic data behind the scenes, and are alerted when candidates exhibit suspicious behaviour. Alerting candidates of this before they begin the assessment, and that their results may be deemed invalid if they do not respond honestly is a useful and effective method of reducing faking. So, as you can see, developing and delivering a valid, reliable and robust psychometric assessment is no mean feat and there are some extremely stringent guidelines attached.

And while this is a burden that we – as psychometric assessment developers – must bear, the great news for employers is that these same stringent guidelines mean you can be confident that tests meeting these requirements will give you accurate, fair and reliable predictions of how candidates will behave and perform at work.

by Jarrah Watkinson