As Registered Psychologists with advanced psychological training and ethical responsibilities, we often come across dodgy situations and claims by other providers who are doing things that we would consider ill advised and even unethical.
We thought we would share with you some of the common myths, lies and half truths we come across in the marketplace:
You can “fake” personality questionnaires so they are not useful!
All good personality tests have measures in them that pick up if someone has tried to “fake” the tool, and as such you know whether the information is valid or not from the assessment. If it is valid it is useful.
Some assessments, such as the California Psychological Inventory (CPI) are very difficult to “fake” and the items are not “face valid”, that is it is not always easy to discern what the question is trying to measure. As such it is not easy to fake, and the incidence of “faked” profiles is relatively rare. Read more about the CPI
Even if the CPI does come back as a “faked” profile, we inform the candidate they need to re-sit the assessment and be more candid as the results are not valid. In most instances, the second profile will be more real and a valid representation of the individuals preferences.
Read more about Faking...
Our tool is the best personality test and unique
- While there are some better and some worse ones, there is no one “best” tool available.
- Certain people may prefer one tool over another, and even well designed tools have strengths and weaknesses.
- People claiming their tool is the best and unique often have a vested interest in claiming this – they make money from selling their tests!
- Strangely enough these same test publishers often are not prepared to have their tool submitted for peer review, where other psychologists and test makers could substantiate their claims or refute them.
- Surely if you have the best tool out there wouldn’t you want your peers to review it and back up your claims….. possibly not if you know your peers are going to disprove your claims.
- For example, one consultancy claims “Saville Consulting develop rigorous and work relevant measurement tools, which have transformed traditional approaches to assessment around the world.” – where is the evidence for this transformation and why has their assessment tool “The Wave” not been submitted for peer review.
- At Niche, we survive in business by selling services from many different tools, not from selling the tools themselves, as such we can give you impartial advice on what we think are the best most useful tools for your situation.
Ipsative (or forced choice) assessments should be used for Recruitment
There are two types of personality assessments Normative and Ipsative assessments.
- An Ipsative test is specifically designed to measure within person differences where they are forced to choose one preference as being stronger than another within themselves.
- Normative tests are a between person measure in that we compare an individual on a preference and how strong this preference is in them compared to others.
Ipsative scores are not “how much of this preference do you possess?” but rather “how much of this preference do you have in relation to your levels of other preferences you possess?”.
On an Ipsative measure you cannot be strong in every preference; if you have more of one preference you have to have less of another.
- To simplify things we will give an analogy in the sports world, Cameron Brown (world class Ironman Triathlete) might say “my strength is in running but I am not as strong when it comes to swimming”.
- On an ipsative measure of his sporting preferences he would come out as being above average on running, average on cycling and below average on swimming, he could not be strong in all three areas.
- However, once we compare Cameron to the rest of the world in swimming he is still very strong and would be well above average, it is only in comparison to his other triathlon disciplines (biking and running) within his own athletic abilities, that you could say swimming is not his strength out of the three triathlon disciplines.
- Under the normative model in theory someone can be above average or even exceptionally high on every scale.
- For Cameron Brown he would be well above average compared to others on all three disciplines in the triathalon and this would be a fair estimate of his abilities compared to others.
In their chapter on the structure, measurement, validity and use of Personality variables in Industrial, Work and Organisational Psychology, Hough and Ones (2001) state:
“Normative measures are useful for comparing people and are thus useful for personnel selection. Ipsative and normative measures are both useful for counselling purposes. In recent years, some have advocated the use of ipsative measures for normative purposes (e.g. selection). We find this inadvisable.”
Our assessment tool predicts success 90% of the time
For the lay person sometimes the claims of less scrupulous test makers or assessment consultants can seem compelling. You see things like “predicts sales success 90% of the time”
- These claims simply are not true, this and other such claims are, at best misleading, at worst fraudulent. There are no assessment tools out there that are that good.
- Unscrupulous test makers and publishers may make claims that look great, but are simply not true.
- Beware of people pushing assessment tools promising the world, there are none that good.
- Assessment tools should be used alongside other information, not as a panacea that eclipses other useful information.
You should only use an Assessment that is Face Valid
- The face validity of an assessment tool concerns the look and feel of the assessment items and whether the person taking the test can see any relevance of the test items or method to the purpose they are being tested for which often is the job or role they are applying for.
- Having high face validity of a test has no relationship to the test actually being predictive of a desired outcome (such as on the job performance) or a valid measure.
- High face validity may improve the candidate comfort with the assessment process, however it does not make it a better tool and it has a big downside in that the assessment will be easier to “fake” or manipulate as the more desirable answer is often obvious.
- For example if you know that teamwork is important in a role what is the right answer to the following highly face valid item: “I like working in a team”?
Only new assessment tools are valuable and good
- An older assessment in psychometric terms has many advantages as the instrument has been reviewed, updated and re-normed more often than a newer tool.
- In fact the greater the age of the instrument, the more opportunity there has been for research into its reliability and validity.
- New tools often have not yet collected much research data, especially on predicative validity, and they may not have appropriate norms available.
EQ is what is important and useful today
EQ is a bit of a fad and there are other fads out there when it comes to assessment tools.
- Emotional Intelligence (EQ) was popularised and became known when a New York writer, Daniel Goleman wrote the book “Emotional Intelligence”.
- In our opinion, EQ as Goleman has popularised it is a fad – psychologists have been measuring the components of EQ since personality questionnaires first became popular in World War I when Woodworth developed an inventory designed to screen out neurotic men who would be unfit for military service.
- The concept of EQ is a “fad” that caught the publics and leader’s attention, the actual things EQ measures are personality characteristics that we have been able to measure accurately for a long time before Goleman came along.
- In addition, the originators of EQ, John Mayer and Peter Salovey, published 2 academic articles on “emotional intelligence” in 1990, well prior to Goleman, and Goleman appears to have stolen the concept and marketed for himself.
- In our opinion, Mayer and Salovey’s research has more merit than Goleman’s as it was the study between the fluid interplay between intelligence and emotions.
- Mayer, Salovey et al (2000) have defined EQ (or EI as they call it) as a type of mental ability, rather than a personality/ability mix.
- We could not agree more with Schmidt, Oh & Shaffer (2016, page 24) in their summary about the construct emotional intelligence quoted here:
“In the last several decades, “emotional intelligence” measures have become popular in many businesses, and emotional intelligence has been treated in the popular press as an actual psychological trait or construct. We have place the label emotional intelligence in quotation marks because most psychologists, including personnel psychologists, do not accept that emotional intelligence is a trait or construct; rather they view emotional intelligence measures as arbitrary amalgamations of items measuring long established psychological traits (cf. Murphy, 2006; Matthews, Zeidner, & Roberts, 2002).”
It is cheaper to take the assessments in-house and do it ourselves
The true costs of in house assessment need to be considered rather than just the fee per assessment when comparing a consulting organisation’s fee against an in-house costs.
In addition to the per assessment cost other considerations include:
- Time – do you have the time to do your assessments in-house? Have you factored in that people tend to only have 2/3rds of their time as productive, and that they have holidays and annual leave where they are unavailable.
- Personnel – do you have enough personnel to take your assessments in-house? Not only that but have you factored in the real cost of the people who manage your assessments in-house and their overheads? Conservative estimates put a person’s overheads at $70K per role, plus there salary, bonuses etc…
- Expertise – do you have the expertise internally to manage the level of assessment you require? If your staff are not doing assessments all the time and do not have the depth of knowledge to fall back on, what is the really quality of information you’re are going to get from your in-house person?
- Quality – what quality of assessments can you buy without professional and qualified assistance? Some tests are only available to registered psychologists.
- Set-up costs – how much do you need to out lay to set up the system and get the materials to use?
- Systems and process costs – do you have the systems in place to manage the assessment process and assessment data to ensure its security and integrity?
- Training costs – How much does it cost to train your personnel on the tests? Should the trained personnel leave, there will be additional training of a new person to be considered.
- Ongoing costs – how much does it cost to buy the tests per person and are there licence fees and other hidden costs. Add in the real costs of having a person or persons doing this and their overheads and resources they need?