Voorspellen van studiesucces in toelatingsprocedures

Selecteren voor het hoger onderwijs
05/07/2019
Susan Niessen
Download
Geen reacties
De Psycholoog Volume 54, Issue 7/8, Pagina 34-45
Summary

NEW RULES, NEW TOOLS: PREDICTING ACADEMIC ACHIEVEMENT IN COLLEGE ADMISSIONS

A. S. M. NIESSEN

Since 2017, admission to higher education programs in the Netherlands happens through selective admission for programs with a fixed number of places, and through non-binding ‘matching’ procedures for open admission programs. This research project aimed to answer the question on what methods to use to predict academic performance within such admission procedures. Scales for noncognitive characteristics such as motivation and personality traits yielded moderate predictive validity when administered in low-stakes conditions, but predictive validity was attenuated when they were administered in a high-stakes context, and were not perceived favourably by applicants. Scores on a curriculum-sampling test, a test that mimics the study program in preparation and content, analogous to work samples in personnel selection, showed high predictive validity, favourable applicant perceptions, and little or no predictive bias based on gender. Hence, curriculum-sampling tests seem to be the best fitting method to use in admission procedures to higher education in the Netherlands.

Log in
Sinds kort worden studenten op basis van selectie- en matchingsprocedures toegelaten tot het hoger onderwijs. Sociaal wenselijke antwoorden maken van scores op motivatie en persoonlijkheid echter slechte voorspellers van studie succes. Proefstuderen zou de meest geschikte invulling van toelatings procedures zijn, concludeert Susan Niessen. Proefstuderen had een hoge predictieve validiteit, werd positief gewaardeerd door kandidaten en liet weinig predictieve bias zien op basis van geslacht.

Introductie

Tot voor kort waren bijna alle studieprogramma’s in het hoger onderwijs toegankelijk voor iedereen die de juiste vooropleiding had voltooid en werd de toelating tot programma’s met een numerus fixus bepaald door een gewogen loting, waarbij de cijfers uit het voortgezet onderwijs de kans op toelating bepaalden. Na enkele jaren van experimenteren worden alle kandidaten voor numerus fixus-studies nu geselecteerd via selectie, waarbij de onderwijsinstellingen de toelatingscriteria bepalen. De overige studieprogramma’s organiseren een verplichte matchingsprocedure of ‘studiekeuzecheck’ die resulteert in een niet-bindend studiekeuzeadvies. Het doel van deze maatregelen is ‘de juiste student op de juiste plaats’ krijgen. Dat zou moeten leiden tot betere studiekeuze, minder uitval, snelle studievoortgang en betere studieprestaties. De vraag is welke instrumenten en procedures het beste ingezet kunnen worden om dit doel te bereiken. In Europa is het meest voorkomende toelatingscriterium de prestatie in het voortgezet onderwijs, meestal in kaart gebracht via het gemiddelde middelbare schoolcijfer. Dat schoolcijfer wordt ook vaak genoemd als de…

Dit artikel is geschreven naar aanleiding van Susan Niessen haar proefschrift New Rules, New Tools: Predicting academic achievement in college admissions, onder supervisie van prof. dr. Rob R. Meijer, dr. Jorge N. Tendeiro en mr. dr. Jaap J. Dijkstra.

Literatuurlijst
  1. Birkeland, S.A., Manson, T.M., Kisamore, J.L., Brannick, M.T. & Smith, M.A. (2006). A meta-analytic investigation of job applicant faking on personality measures. International Journal of Selection and Assessment, 14, 317–335. doi:10.1111/j.1468–2389.2006.00354.x.
  2. Callinan, M. & Robertson, I.T. (2000). Work sample testing.International Journal of Selection and Assessment,8, 248-260. doi:10.1111/1468- 2389.00154
  3. Cleary, T.A. (1968). Test bias: Prediction of grades of negro and white students in integrated colleges. Journal of Educational Measurement, 5, 115–124. doi:10.1111/j.1745-3984.1968.tb00613.x
  4. Credé, M. & Kuncel, N.R. (2008). Study habits, skills, and attitudes: The third pillar supporting collegiate performance. Perspectives on Psychological Science, 3, 425-453. doi:10.1111/j.1745-6924.2008.00089.x
  5. Cremonini, L., Leisyte, L., Weyer, E. & Vossensteyn, J. J. (2011). Selection and matching in higher education: An international comparative study. Enschede, the Netherlands: Center for Higher Education Policy Studies (CHEPS).
  6. Christiansen, N.D., Burns, G.N. & Montgomery, G.E. (2005). Reconsidering forced-choice item formats for applicant personality assessment.Human Performance,18, 267-307. doi:10.1207/ s15327043hup1803_4
  7. de Visser, M., Fluit, C., Fransen, J., Latijnhouwers, M., Cohen-Schotanus, J. & Laan, R. (2017). The effect of curriculum sample selection for medical school. Advances in Health Science Education, 22, 43–56. doi:10.1007/ s10459-016-9681-x
  8. Dienst Uitvoering Onderwijs (2014). Jaarverslag numerus fixus-opleidingen. Opgevraagd via https://tinyurl.com/y94c8ef3
  9. Goho, J. & Blackman, A. (2006). The effectiveness of academic admission interviews: An exploratory meta-analysis.Medical Teacher,28, 335-340. doi:10.1080/01421590600603418
  10. Griffin, B. & Wilson, I.G. (2012). Faking good: Self-enhancement in medical school applicants. Medical Education, 46, 485–490. doi:10.1111/j.1365- 2923.2011.04208.x.
  11. Hausknecht, J.P., Day, D.V. & Thomas, S.C. (2004). Applicant reactions to selection procedures: An updated model and meta-analysis. Personnel Psychology, 57, 639-683. doi:10.1111/j.1744-6570.2004.00003.x
  12. Hunter, J.E., Schmidt, F.L. & Le, H. (2006). Implications of direct and indirect range restriction for meta-analysis methods and findings. Journal of Applied Psychology,91, 594-612. doi:10.1037/0021-9010.91.3.594
  13. Hoover, E. (2013). Noncognitive measures: The next frontier in college admissions.Chronicle of Higher Education. Opgevraagd via: https://www.collegesuccessfoundation.org/document.doc?id=851
  14. Keiser, H.N., Sackett, P.R., Kuncel, N.R. & Brothen, T. (2016). Why women perform better in college than admission scores would predict: Exploring the roles of conscientiousness and course-taking patterns.Journal of Applied Psychology, 101, 569-581. doi:10.1037/apl0000069
  15. Kruschke, J.K., Aguinis, H. & Joo, H. (2012). The time has come: Bayesian methods for data analysis in the organizational sciences. Organizational Research Methods, 15, 722-752. doi:10.1177/1094428112457829.
  16. Kruschke, J.K. & Liddell, T.M. (2018). The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective. Psychonomic Bulletin & Review, 25, 178–206. https://doi.org/10.3758/s13423-016-1221-4
  17. Kyllonen, P. & Bertling, J. (2017, April). Interpersonal and intrapersonal skills assessment: Design, development, scoring, and reporting. Workshop provided at the Annual Meeting of the National Council on Measurement in Education, San Antonio, Texas.
  18. Lievens, F. (2013). Adjusting medical school admission: Assessing interpersonal skills using situational judgement tests. Medical Education, 47, 182-189. doi:10.1111/medu.12089
  19. Lievens, F. & Coetsier, P. (2002). Situational tests in student selection: An examination of predictive validity, adverse impact, and construct validity. International Journal of Selection and Assessment,10, 245–257. doi:10.1111/1468-2389.00215
  20. Mattern, K. Sanchez, E. & Ndum, E. (2017). Why do achievement measures underpredict female academic performance? Educational Measurement: Issues and Practice, 36, 47-57. doi:10.1111/emip.12138
  21. Morgeson, F.P., Campion, M.A., Dipboye, R.L., Hollenbeck, J.R., Murphy, K. & Schmitt, N. (2007b). Reconsidering the use of personality tests in personnel selection contexts. Personnel Psychology, 60, 683–729. doi:10.1111/j.1744-6570.2007.00089.x.
  22. Murphy, S.C., Klieger, D.M., Borneman, M.J. & Kuncel, N.R. (2009). The predictive power of personal statements in admissions: A meta-analysis and cautionary tale. College and University,84, 83-86.
  23. Naylor, J.C. & Shine, L.C. (1965). A table for determining the increase in mean criterion score obtained by using a selection device. Journal of Industrial Psychology, 3, 33-42.
  24. Ones, D.S., Dilchert, S., Viswesvaran, C. & Judge, T.A. (2007). In support of personality assessment in organizational settings. Personnel Psychology, 60, 995-1027. doi:10.1111/j.1744-6570.2007.00099.x
  25. Patterson, F., Knight, A., Dowell, J., Nicholson, S., Cousans, F. & Cleland, J. (2016). How effective are selection methods in medical education? A systematic review. Medical Education, 50, 36–60. doi:10.1111/medu.12817
  26. Pauls, C.A. & Crost, N.W. (2004). Effects of faking on self-deception and impressionmanagement scales. Personality and Individual Differences, 37, 1137–1151. doi:10.1016/j.paid.2003.11.018.
  27. Reibnegger, G., Caluba, H.C., Ithaler, D., Manhal, S., Neges, H.M. & Smolle, J. (2010). Progress of medical students after open admission or admission based on knowledge tests. Medical Education, 44, 205–214. doi:10.1111/j.1365-2923.2009.03576.x
  28. Resing, W.C.M. & Drenth, P.J.D. (2007). Intelligentie: Weten en meten. Amsterdam: Uitgeverij Nieuwezijds.
  29. Richardson, M., Abrahams, C. & Bond, R. (2012). Psychological correlates of university students’ academic performance: A systematic review and meta-analysis. Psychological Bulletin, 138, 353–387. doi:10.1037/ a0026838.
  30. Robbins, S.B., Lauver, K., Le, H., Davis, D., Langley, R. & Carlstrom, A. (2004). Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130, 261-288. doi:10.1037/0033-2909.130.2.261
  31. Sackett, P.R., Kuncel, N.R., Arneson, J.J., Cooper, S.R. & Waters, S.D. (2009). Does socioeconomic status explain the relationship between admissions tests and post-secondary academic performance? Psychological Bulletin, 135, 1–22. doi:10.1037/a0013978
  32. Schmidt, F.L. & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262–274. doi:10.1037/0033-2909.124.2.262
  33. Steiner, D.D. & Gilliland, S.W. (1996). Fairness reactions to personnel selection techniques in France and the United States. Journal of Applied Psychology, 81, 134-141. doi:10.1037/0021-9010.81.2.134
  34. Taylor, H.C. & Russell, J.T. (1939). The relationship of validity coe§ cients to the practical effectiveness of tests in selection: discussion and tables. Journal of Applied Psychology, 23, 565-578. doi:10.1037/h0057079
  35. van den Broek, A., Nooij, J., van Essen, M. & Duysak, S. (2017). Selectie & plaatsing bij numerusfi xusopleidingen. Nijmegen, the Netherlands: ResearchNed. Opgevraagd via: https://tinyurl.com/y9449rhr
  36. van der Maas, H. & Visser, K. (2017, June 8). Wet selectie studenten is niet uitvoerbaar. De Volkskrant. Opgevraagd via: https://tinyurl.com/yak8fxft
  37. Vihavainen, A., Luukkainen, M., & Kurhila, J. (2013, October). MOOC as semester-long entrance exam. Paper presented at the 14th Annual ACM SIGITE Conference on Information Technology Education, Orlando, Florida, United States.
  38. Visser, K., van der Maas, H., Engels-Freeke, M. & Vorst, H. (2012). Het effect op studiesucces van decentrale selectie middels proefstuderen aan de poort. Tijdschrift voor Hoger Onderwijs, 30, 161–173.
  39. Warps, J., Nooij, N., Muskens, M., Kurver, B. & van den Broek, A. (2017). De studiekeuzecheck. Nijmegen, the Netherlands: ResearchNed. Opgevraagd via: https://tinyurl.com/y8dnvn3s
  40. Wernimont, P.F. & Campbell, J.P. (1968). Signs, samples, and criteria. Journal of Applied Psychology, 52, 372-376. doi:10.1037/h0026244
  41. Westrick, P.A., Le, H., Robbins, S.B., Radunzel, J.R. & Schmidt, F.L. (2015). College performance and retention: A meta-analysis of the predictive validities of ACT scores, high school grades, and SES. Educational Assessment, 20, 23-45. doi:10.1080/10627197.2015.997614
  42. Wet kwaliteit in verscheidenheid. Memorie van Toelichting. Opgevraagd via: https://tinyurl.com/y7ekn87z