Queens University Belfast

Queen’s University Belfast

Replicated Multiple Mini Interview (MMI) online.
Ideal for International Applications.

In terms of our international recruitment, using Sonru I’d say we’re ahead of the march in our ability to be able to identify the best people. Professor Donald Burden, Director of the Centre for Dental Education, Queen’s University Belfast

About Queen's Centre for Dental Medicine

The Centre for Dental Education is responsible for the organisation, management and delivery of the Dental course at Queen's. The School of Dentistry has been responsible for delivering dental education in Northern Ireland since 1920. The five-year undergraduate course in Dentistry is based on the requirements of the UK’s General Dental Council (GDC) and was ranked number one in UK by the Times Good University Guide in 2008.

Context

Two years ago, the Centre for Dental Education began using the multiple mini interview (MMI), a structured interview format the purpose of which is to assess applicants’ non-cognitive skills such as: maturity and responsibility, interpersonal and communication skills, confidence, self awareness, empathy, ethical and moral values, and awareness of the demands of training as a dental surgeon. This year, the Bachelor of Dental Surgery (BDS) programme opened 15 places to international students (non-UK/EU). The Centre for Dental Education used Sonru to replicate the Multiple Mini Interview process online to standardise the interview process for all applicants (UK/EU and non-EU).About the Multi Mini Interview (MMI)The MMI system was first developed by the School of Medicine in McMaster University Canada to address two widely recognized problems. First, it has been shown that traditional interview formats or simulations of educational situations do not accurately predict performance in medical school. Secondly, when a licensing or regulatory body reviews the performance of a dental or medical professional subsequent to patient complaints, the most frequent issues of concern are those of the non-cognitive skills, such as interpersonal skills, professionalism and ethical/moral judgment.Queen’s MMI interview process consists of a series of mini interviews where candidates are presented with a 'scenario' based question.  The candidates meet eight interviewers (all members of the Dental School teaching team) in pairs across 4 different rooms. Each interviewer asks one question and the candidate is given 4 minutes to answer.  There is absolutely no prompting/interaction during the interview.  All students are scored and ranked in the same way; the applicants with the highest scores receive an offer. There shouldn’t be any significant variance between the scores from each of the eight interviewers and the scores are tested for reliability using Cronbach's α – a coefficient of reliability commonly used as a measure of the internal consistency or reliability.

Automating the Multi Mini Interview (MMI)According to Professor Burden, Director of the Centre for Dental Education, who championed the Sonru project, “Sonru worked very well, we presented the scenario based questions in written format and allocated each respondent with 4 minutes to answer the question”. In order to reproduce the same environment, the scoring was done exactly the same as it is done face-to-face i.e. the interviewers were shown the video interviews in pairs and given the same amount of time to allocate scores. The scores were also tested for scientific reliability.

Initial Fears allayed through usage

When the staff of the Centre for Dental Education were informed that Sonru’s video interviewing solution was going to be used to automate the MMI process for non-EU applicants, two key concerns emerged. The first concern related to an increased workload but the Administrative Manager Helen Martin was pleasantly surprised ‘how easy it was to organize and excellent from the administrative perspective’. Interviewer Orlagh Hunt agreed and ‘could see a huge benefit for backroom staff as they had less to do in terms of arranging the interviews and meeting and greeting interviewees etc.’. Secondly, both the administrative staff and interviewers were concerned that Sonru would be difficult to use but unanimous positive feedback with respect to ease-of-use of the Sonru system was recorded. Interviewers and administrators found it ‘extremely easy to use, completely served the purpose and worked well’, ‘very straightforward’ and were ‘surprised how easy it was, it didn’t take up much time.’

Experiencing and Grading the Candidate

What was uniquely useful about Sonru was that it was a stress test in itself which is relevant for our admittance procedures. Dentistry is a stressful occupation so an applicant’s handling of the Sonru system may be indicative of how they might handle other stresses. Professor Burden, Director of the Centre for Dental Education, Queen’s University Belfast

As the objective of the MMI process is to assess non-cognitive skills particularly communication skills, there were concerns that the video interview format might not be appropriate. Interviewer Orlagh Hunt worried that "we’d miss communication skills using Sonru but it was very apparent watching the video responses if they [candidates] were looking at the screen or looking away or whether they were engaged”. Fellow interviewer Susan Morison agreed adding from the interviewer’s perspective that the system ‘gives you the opportunity to concentrate on what they [the candidates] are saying’.

Sonru was better than face-to-face from a marker’s perspective, the ability to grade answers worked and was quite objective Susan Morison, Educationalist, Centre for Dental Education, Queen’s University Belfast

Furthermore, the Interviewers felt the automated interview was more advantageous for candidates. Interviewer Orlagh Hunt added ‘in some ways the Sonru version worked better, you don’t feel bad writing or making notes while the interviewee was speaking.’Flexibility without Compromising Structure

The date and time to view the interviews wasn’t set in stone, you could rearrange it with your Interview partner whereas if you missed the days of face-to-face interviews it would make the MMI scoring difficult, the process wouldn’t be effective as the Interviewers would not be uniform. Orlagh Hunt, Educationalist, Centre for Dental Education, Queen’s University Belfast   Although the MMI interview process has a very rigid structure, the ability to view the video interviews online offered enormous flexibility to the eight Interviewers. Professor Burden welcomed the convenience of enabling the Interviewers to ‘fit it around their schedules – they either came in early for work or stayed a little later to get it done without upsetting their own work’ instead of the traditional method which takes ‘1/3 of my clinical staff away from their work for 3 days’.

Ideal for International (non-EU) Applications

Professor Burden found ‘Sonru ideal for interviewing International applicants’. Without automated video interviewing, typically colleges send 2 members of staff to a particular location and as a result are limited to interviewing applicants only from that region and it is impossible to administer the MMI unless you fly out the entire Interviewing team. Professor Burden added ‘

It also allows us to reject in a less difficult environment, as in the student has not incurred the expense and time in traveling thousands of miles to sit our interview’. The Interviewers agree that Sonru was ideal MMI interview solution for international applications.

It’s a much better option than bringing students over or sending interviewers out as it enables us to use exactly the same process as the UK/EU students Susan Morison, Educationalist, Centre for Dental Education, Queen’s University Belfast   Orlagh Hunt added ‘the great thing about this is that every Non-EU applicant was doing a Sonru interview so it was standardized, it wouldn’t be fair to have some applicants doing face-to-face and others automated’. Sonru offers an edge to those in the competitive world of dental and medical education as Professor Burden maintains ‘in terms of our international recruitment, using Sonru I’d say we’re ahead of the march in our ability to be able to identify the best people’.

Conclusion

We do a lot of analysis of our interview processes and are quite pleased with our assessments. We are confident the Multiple Mini interview has transferred onto the Sonru system. It suits us very well. Professor Donald Burden, Director of the Centre for Dental Education, Queen’s University Belfast

Sonru worked very well for Queen’s University’s Dental School with the ability to successfully replicate the structured Multiple Mini Interview (MMI) online to ensure a standard screening interview process for applicants all over the world.

‘It has reduced staff costs and time but also delivers a robust and reliable interview system’ concludes Professor Burden, Director of the Centre for Dental Education, Queen’s University Belfast