Ri five minutes with: Oliver Savill, Director, Test Partnership

Ri five minutes with: Oliver Savill, Director, Test Partnership


MindmetriQ is a new series of validated gamified assessments. The publisher, Test Partnership, claim that it’s more accurate, more reliable, and more engaging than traditional psychometric testing. We caught up with Director Oliver Savill to find out more.

Tell us about MindmetriQ

MindmetriQ is a series of validated gamified assessments which use serious science. Test Partnership developed MindmetriQ because they were frustrated with the lack of scientific rigour being applied to other gamified assessments coming onto the market. From inception MindmetriQ has been developed according to industry best-practice psychometric test development. Accordingly, MindmetriQ has been professionally validated and has high reliability.

The six games in MindmetriQ all measure cognitive ability, which has been shown to correlate well with job performance.

For increased reliability, better test security, and improved candidate experience MindmetriQ uses computer adaptive testing (CAT) to deliver questions precisely targeted to each candidate.

What is CAT and how does MindmetriQ use it?

Most other gamified assessments (and many psychometric tests generally) give each candidate the same set of questions. However MindmetriQ uses CAT, which picks questions of the optimal difficulty from a large item bank of questions. As the candidate progresses through the assessment, the CAT algorithm calculates the candidate’s ability in real-time and serves the best question based on their estimated ability. This means MindmetriQ can achieve higher levels of test reliability whilst simultaneously taking less time to administer (each of the MindmetriQ assessments take between four and seven minutes to complete).

Since MindmetriQ uses CAT, it is significantly more accurate than non-CAT assessments, while also allowing us to shorten the assessments without sacrificing reliability.

Do your Gamified Assessments work as well as Traditional Assessments?

Gamification allows us to increase task complexity far beyond what static assessment formats normally allow, which improved the measurement of cognitive ability. So MindmetriQ offers a more rich cognitive assessment. Our research suggests that each assessment within the MindmetriQ series displays comparable validity and reliability to a traditional ability test, but takes a fraction of the typical administration time. Because the MindmetriQ series comprises six assessments (compared to say three tests in a traditional battery of aptitude series), the full series of MindmetriQ displays superior psychometric properties to traditional ability tests and can be completed in less time.

How do Gamified Assessments improve candidate experience?

It’s no secret that traditional ability tests aren’t everyone’s idea of fun. This is why gamified assessments have become so popular. They can significantly improve candidate experience through making the process more enjoyable and engaging. Not only does this make the experience less onerous for the candidate, but it can help reduce serious test anxiety, which hinders performance in otherwise high potential candidates.

Does MindmetriQ help with diversity and inclusion?

MindmetriQ has undergone full validation and trialling, and adverse impact effect sizes for protected groups are all non-significant. Extensive research supports the fairness of MindmetriQ. During the development of MindmetriQ we trialled thousands of questions and only those shown not to introduce adverse impact were selected for the item bank.

MindmetriQ is fully mobile enabled, which is important when increasing socioeconomic diversity. Traditional assessments almost always require either a desktop computer or a laptop to complete, discouraging applications from those who do not have readily available access to either of those devices. MindmetriQ however can be completed on a mobile device, making participation easily accessible, and helping to level the playing field.

How do you prevent cheating?

Most gamified assessments employ rudimentary psychometric science, administering the exact same set of questions or puzzles to every candidate. Therefore, it’s quite easy to cheat on these assessments as solutions are often readily shared online, making these assessments completely unsuitable for unsupervised online testing. MindmetriQ however uses CAT, reducing the likelihood that different candidates will ever receive the same set of questions. Since no two candidates are likely to see the same series of questions, it’s very difficult to cheat on MindmetriQ.

Do you use black-box algorithms in your scoring?

Ironically, many assessment providers brag that their assessments record “ten thousand data points”, “one million data points”, even “ten million data points”, shamelessly violating the statistical principle of parsimony. In reality these datapoints are meaningless numbers, serving only to cover-up the fact their games don’t actually measure anything of value. Scores on MindmetriQ are based solely on test performance, ensuring that: a) scores aren’t corrupted by unnecessary random data; and b) candidates aren’t second-guessing what actions are recorded and used against them, which allows candidates to focus solely on their performance.  We don’t use ‘black box’ algorithms and we’re happy to talk anyone through how the science behind MindmetriQ.

More information can be found here: https://www.testpartnership.com/gamified-assessment.html, where you will also be able to take part in some gamified assessments.


Thursday, 14 February 2019

Share this article

Any questions? Any comments?

Your instant reactions to this article can be posted here.

Be the first to make a comment...