IMJ
About IMJ
Disclaimer
Contact IMJ
Register as Reviewer
Register As Author
View IMJ Authors
View IMJ Volumes
View Supplement
Issue Archive 1980-1996
Subscription Detail 2010

IMJ Search

 

Advanced Search
 
 
General Practitioners’ Perspectives on Revised Entry and Selection Methods to Medicine and the HPAT   Back Bookmark and Share
Thomas Dennehy,Maureen E Kelly,Siun O'Flynn
Ir Med J. 2013 Apr;106(4):113-5
T Dennehy1, M Kelly2, S O`Flynn3
1
South West Specialist Training Programme in General Practice
2Discipline of General Practice, NUI Galway
3Medical Education Unit, School of Medicine, University College Cork, Cork

Abstract
Revised selection mechanisms to medicine now include an adjunct admission test known as the HPAT. Stakeholder acceptability is a key issue. General Practitioners (GPs) represent an important stakeholder group. A questionnaire was administered to a regional sample of GPs evaluating their knowledge of the new selection system, views on acceptability, performance on sample HPAT questions and perceptions of their relevance. 92 respondents (75.4%) reported they had little or no knowledge of the HPAT. GPs supported the use of aptitude tests (85 respondents 69.7%). However almost one third (39 respondents, 32%) either disagreed or strongly disagreed with the reforms introduced. The majority strongly supported the Leaving Certificate as a selection tool (118 respondents, 96.7%). GPs performed well in the sample questions. Items from Section 2, measuring interpersonal understanding, were judged to be the most relevant.

 

Introduction
The introduction in Ireland of a revised process to determine admission to medical school has been contentious.1,2 Reforms in 2009 included the introduction of the HPAT (Health Professions Admission Test) coupled with an adjustment of the weighting applied to the Leaving Certificate and a moderation of the calculation of Leaving Certificate points.3 The HPAT is a 2 ½ hour multiple choice paper. It comprises three sections; 1. Logical Reasoning and Problem Solving; 2. Interpersonal Understanding and 3. Non-Verbal Reasoning4. It belongs to a family of selection tools that test general mental ability. Other similar such tests include the UMAT, used in Australia and the UKCAT which was introduced to Great Britain in 20065,6. Research in the area of entry and selection to medical school is growing internationally. There is an increasing acceptance that the more traditional approaches to selection such as sole reliance on academic achievement or traditional interview are of themselves insufficient to select tomorrow’s doctors7.   Much of the research on tests similar to HPAT examines the test predictive validity. Overall, the findings are mixed in terms of whether such tests predict subsequent medical school performance or enhance the selection process8,9.

It is important however that specialised admission tests are acceptable to stakeholders and that the assessment items therein appear relevant i.e. achieve face validity. To date this important question has received less attention. Previously the performances of Consultant Surgeons, NCHDs and Medical Students in a modified HPAT were analysed10. In this study no results of statistical significance were returned, however the performance of consultant surgeons was only minimally better than undergraduate medical students. This caused the authors to question the acceptability of the test. In a separate study the Leaving Certificate and a modified HPAT were both found to be predictive of college exam performance; however they were not correlated with each other leading the authors to conclude that were assessing different applicant attributes11.  General Practitioners are an important and influential stakeholder group representing the career pathway chosen by approximately 50% of all future medical school entrants. We were keen to ascertain the level of knowledge amongst GPs regarding the new system for admission into medicine; establish their views on its acceptability; determine their performance in sample questions and evaluate their perception of the relevance of test items used.
Methods
This was a quantitative observational study, using a 27 item questionnaire and three sample HPAT questions. Section 1 determined demographics; Section 2 assessed knowledge of the HPAT and the revised selection criteria; Section 3 established GPs’ views on acceptability of the HPAT, the Leaving Certificate and other tools for the selection of medical students and, Section 4 comprised three sample HPAT questions which the GPs answered and subsequently rated on a four point likert scale in terms of relevance to medical practice. Questionnaires were posted, with a participant information letter and consent form, to a convenience sample of 143 GPs from the Cork Registry (ICGP Directory) and to the 12 GP trainees of the Cork Specialist Training Programme in General Practice, in year two at the time of the questionnaire (September 2010). Telephone reminders to non responders were conducted one week later.  Ethical approval was obtained from the Research Ethics Committee of the Cork Teaching Hospitals (CREC). All data was entered and analysed using SPSS 17.0 for Windows (SPSS, Inc., Chicago, IL, USA).



Results
Demographics
A response rate of122 (79%) was achieved comprising 114 GPs and 8 GP trainees of which there were 67 males and 54 females (one respondent did not complete this section). Ages ranged from 30 to 69 years. The majority of respondents were aged between 30-49years.
Figure 1


Knowledge of HPAT
Respondents were asked to rate their own knowledge of the HPAT on a seven point likert scale. The majority 92 respondents, (75.4%) reported they had little or no knowledge of the HPAT while only 4 respondents, (3.3%) felt they had good knowledge.  GPs seemed unaware of who designs the HPAT paper annually (40 respondents, 33%).  The majority, 96 respondents, (78.7%), also seemed unsure of how entry points were calculated based on combined HPAT and Leaving Certificate scores.     

Views on acceptability of HPAT and other selection tools
These are summarised in table 1. A majority n=85 or (69.7%) indicated that aptitude testing is an appropriate tool for determining entry to medicine.  Almost all respondents believe that the Leaving Certificate is an appropriate selection tool (118 respondents,96.7%).  Other selection mechanisms were also considered appropriate, such as interviews (67 respondents, (54.9%)) and personality traits (63 respondents, (51.6%)). A significant number of respondents either disagreed or strongly disagreed with using personal statements (64 respondents, 52.5%)) and knowledge of the course; (45 respondents, 36.9%) in the selection process. However when asked to directly compare old versus new entry and selection mechanisms (i.e. if the newer selection criteria were more effective than the Leaving Certificate alone in identifying the most appropriate candidates to enter medical school), almost one third either disagreed or strongly disagreed (39 respondents, 32%), while only 25 agreed or strongly agreed (20.5%). This is further illustrated in Figure 1.

Respondents appear to acknowledge that socioeconomic factors can affect test outcomes but that the Leaving Certificate may be marginally more influenced by this variable than the HPAT. When we asked separately if performance in each were affected by socioeconomic status; 87, (71.3%) agreed on some level that it did affect the Leaving Certificate performance, while 80, (65.6%) were of the opinion that it affected HPAT performance.


Sample HPAT questions
Table 1 captures the pattern of correct responses to sample HPAT questions, one from each of the three Sections of a HPAT-Ireland practice test booklet and the estimation of question relevance to future medical practice, (section 2 was a two part question). While 69 respondents (56.6%) identified the correct answer to question 1, from Section 1(Logical Reasoning and Problem Solving), the majority felt it was irrelevant as a question type in determining suitability to practice medicine. Respondents were more successful in the second sample question representing Section 2 (Interpersonal Understanding), with 103 (84.4%) respondents identifying the correct answer in part one, and 81 respondents (66.4%) in part two, and more positive feedback regarding the relevance of such a question. The results of the third sample question representing Section 3 (Non-Verbal Reasoning) were similar to the trends observed in section one with 84 (68.9%) respondents identifying the correct answer and a majority questioning its relevance.
Discussion
The strength of this work is that it evaluates the knowledge and opinion of the new entry and selection mechanisms of a sample of regional GPs. A high response rate was achieved reflecting the keen level of interest in this selection process. There is no reason to think that these responses do not reflect those of GPs elsewhere. There are however also obvious limitations to this study; the sample size was relatively small and represents a convenience sample. Respondents answered only three HPAT test questions, hence, findings cannot be generalised to the entire HPAT paper. The study design was quantitative which limits the depth of data that our questionnaire was able to elicit from our participants.  Therefore results need to be interpreted with caution.

The GPs in this survey report a knowledge gap with respect to the HPAT and the detail of the revised selection criteria. Over half of all the GPs stated that they had no knowledge of the HPAT and over three quarters were unsure of how entry points were calculated according to the HPAT / Leaving Certificate combined system. This is to be expected as the reforms are recent and there was limited occasion for consultation within the medical profession before their introduction. However, as the attrition rate in medical school is low, the selection process effectively determines who will become the doctors of tomorrow12. Therefore it is desirable that all medical professionals have adequate opportunity to be well informed regarding the selection tests used. GPs appear relatively divided as regards the merits of the revised approach to entry and selection to medicine. While over one fifth strongly disagree with the revised mechanisms it would seem that there also was some dissatisfaction with the previous system as only one tenth strongly prefer the older approach. This would suggest that many are relatively undecided or open minded and may be influenced by emerging evidence or perhaps personal experience. 

GPs appear to embrace the use of selection tools other than knowledge-based tests such as the Leaving Certificate and many seemed to be in favour of the use of interviews and personality trait testing. The support for interviews may be influenced by recent innovations in that format or experience with these at post graduate level however the acceptability of personality trait testing is less validated.13,14 Although clearly the Leaving Certificate system is still endorsed as the most appropriate metric, a significant number of respondents were amenable to the use of aptitude tests. This prompts the consideration that discontent with the new system may not necessarily be related to the use of HPAT or similar tests per se, but rather the actual content of the test and the mechanism by which their use is implemented. Other unidentified issues may be at play. Many GPs correctly identify the degree to which socioeconomic status can affect scores in school exit examinations like the Leaving Certificate and this is a cause of concern elsewhere15. They express concerns however that the HPAT may also favour the socioeconomically advantaged. Equity of access is a key issue in entry and selection and potential socioeconomic advantages associated with tests must be carefully evaluated. The majority of GP respondents identify correct responses to HPAT sample test material however question the relevance of some of the items. Although HPAT sample questions cannot be representative of the entire test material, it is interesting that Section 2 which sets out to measure interpersonal understanding currently achieves most acceptability amongst general practitioners. Perhaps expansion of the representation of this type of test material would lead to greater acceptability of tests like HPAT.

In conclusion, medical school entry and selection will undoubtedly continue to generate debate. This is appropriate as the outcomes affect all. Our study suggests that although support for traditional knowledge based tests is strong, additional admission tests which measure non cognitive domains, especially those related to interpersonal skills, are also acceptable to stakeholders.
Correspondence: S O’Flynn
Department of Medical Education, University College Cork, Cork
Email: siun.oflynn@ucc.ie
References 
1. Donnelly K, Heffernan B Medicine points still out of reach as aptitude test deemed a failure. Irish Independant.22-08-2011. http://www.independent.ie/lifestyle/education/latest-news/medicine-points-still-out-of-reach-as-aptitude-test-deemed-a-failure-2854097.html accessed 29/05/2012
2. Murray N.  More students with lower leaving cert points get medicine. Irish Examiner.23-08-2011.http://www.irishexaminer.com/ireland/more-students-with-lower-leaving-cert-points-get-medicine-165133.html accessed 29/05/2012
3. Central Applications Office (CAO) Selection criteria for Undergraduate Entry to Medicine for EU Applicants 2012 http://www2.cao.ie/downloads/documents/2012UGMedEntry.pdf accessed 29/05/2012
4. ACER Health Professions Admission Test – Ireland http://www.hpat-ireland.acer.edu.au/ accessed 29/05/2012
5. ACER Undergraduate Medicine and Health Sciences Admission Test http://umat.acer.edu.au accessed 29/05/2012
6. UKCAT Consortium The UK Clinical Aptitude Test http://www.ukcat.ac.uk/ accessed 29/05/2012
7. Patterson F, Ferguson E “Selection for medical education and training” in T. Swanwick Understanding Medical Education: Evidence, Theory and Practice Oxford, UK, Wiley-Blackwell 2010.Chp 24. 
8. James D, Yates J, Nicholson S. Comparison of A level and UKCAT performance in students applying to UK medical and dental schools in2006: cohort study. BMJ.2010;340:c478.
9. Wright SR, Bradley PM. Has the UK Clinical Aptitude Test improved medical student selection? Medical education. 2010 Nov;44:1069-76.
10. Quinn, A, Corrigan, M A,Broderick, J, McEntee, G, Hill., A comparison of performances of consultant surgeons, NCHDs and medical students in a modified HPAT examination. Ir Med J. 2010; 103.172-3
11. Halpenny D, Cadoo K, Halpenny M,Burke J, Torreggiani WC. The Health Professions Admission Test (HPAT) score and leaving certificate results can independently predict academic performance in medical school: do we need both tests? Ir Med J. 2010;103;10;300-2
12. Eva KW, Reiter HI Where judgement fails: pitfalls in the selection process for medical personnel Adv Health Sci Educ Theory Pract 2004; 9;2:161-74
13. Lumsden, M. A., Bore, M., Millar, K., Jack, R., Powis, D.  Assessment of personal qualities inrelation to admission to medical school Med Educ 2005;39; 3; 258-65
14. Eva KW, Rosenfeld J, Reiter HI, Norman GR. An admissions OSCE: the multiple mini-interview. Med Educ. 2004 Mar;38:314-26.
15. The demography of medical schools: a discussion paper. London: BMA; 2004.

Author's Correspondence
No Author Comments
Acknowledgement
No Acknowledgement
Other References
No Other References
   
© Copyright 2004 - 2009 Irish Medical Journal