Psychometric Validation of A Research Skills Assessment For Pre-Service Science Teachers Using The Rasch Model

Aldeva Ilhami, Topik Hidayat, Riandi riandi, siti sriyati

Abstract


This study aims to validate a research‐skills assessment instrument for pre-service science teachers using Rasch modeling to evaluate reliability, item fit, and unidimensionality. The instrument was designed to measure the extent of research skills among undergraduate students in science education programs. A cross-sectional survey was conducted with 114 participants recruited through quota sampling from several universities in Riau, Indonesia. Respondents completed a 40-item research-skills instrument. Data were analyzed with the Rasch model using Winsteps version 3.73. The results indicate satisfactory internal consistency (Cronbach’s alpha = 0.75), excellent item reliability (0.97), and acceptable person reliability (0.75). Rasch diagnostics further support the unidimensionality of the instrument, indicating that the items predominantly measure a single latent construct of research skills. These findings provide evidence for both the reliability and construct validity of the instrument within this population. Practically, the validated instrument can be used by higher education institutions to diagnose and monitor the research competencies of pre-service science teachers and to evaluate the effectiveness of research-oriented instructional interventions. By employing Rasch analysis, this study enhances the precision and defensibility of measurement in the context of research-skills assessment in Indonesia, offering a foundation for ongoing refinement and wider application in similar educational settings


Full Text:

PDF

References


Asghar, A. U. R., Aksoy, M., Graham, A. I., & Baseler, H. A. (2023). Developing research skills in medical students online using an active research study. BMC Medical Education, 23(1), 1–13. https://doi.org/10.1186/s12909-023-04781-5

Bond, T. G., Yan, Z., & Heene, M. (2021). Applying the Rasch Model:Fundamental Measurement in The Human Science. In Applying the Rasch Model (Fouth Edit). Routledge. https://doi.org/10.4324/9781410614575

Boone, W. J. (2016). Rasch analysis for instrument development: why, when, and how? CBE—Life Sciences Education, 15(4), rm4.

Daryanes, F., Zubaidah, S., Mahanal, S., & Sulisetijono, S. (2025a). Development and Validation of Research Skills Instruments for Pre-service Biology Teachers. Participatory Educational Research, 12(2), 75–96. https://doi.org/10.17275/per.25.20.12.2

Habash, R. (2024a). Engaging Undergraduate Engineering Students in a Research and Design Experience. 2024 47th ICT and Electronics Convention, MIPRO 2024 - Proceedings (pp. 1300–1305). Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/MIPRO60963.2024.10569681

Heikkilä, M., Hermansen, H., Iiskala, T., Mikkilä-Erdmann, M., & Warinowski, A. (2023). Epistemic agency in student teachers’ engagement with research skills. Teaching in Higher Education, 28(3), 455–472.

Hidayat, T., Rustaman, N., & Siahaan, P. (2021). Developing Students’ Research Skills with Adapted Primary Literature. Thabiea: Journal of Natural Science Teaching, 4(2), 121–135.

Hordern, J. (2019). Knowledge, evidence, and the configuration of educational practice. Education Sciences, 9(2), 70.

Ješková, Z., Lukáč, S., Šnajder, Ľ., Guniš, J., Klein, D., & Kireš, M. (2022). Active Learning in STEM Education with Regard to the Development of Inquiry Skills. Education Sciences, 12(10). https://doi.org/10.3390/educsci12100686

Katayev, Y. S., & Burdina, E. I. (2023). Developing research competencies in secondary school teachers within the context of contemporary educational content. Educacao e Pesquisa, 49. https://doi.org/10.1590/S1678-4634202349257253EN

Khine, M. S. (2020a). Rasch measurement. Springer.

Leupen, S. M., Kephart, K. L., & Hodges, L. C. (2020). Factors influencing quality of team discussion: Discourse analysis in an undergraduate team-based learning biology course. CBE—Life Sciences Education, 19(1), ar7.

Maddens, L., Depaepe, F., Janssen, R., Raes, A., & Elen, J. (2020). Evaluating the Leuven Research Skills Test for 11th and 12th Grade. Journal of Psychoeducational Assessment, 38(4), 445–459. https://doi.org/10.1177/0734282918825040

Mdikana, A. A. (2025). The role research plays in addressing challenges in higher education and in society. In Social Implications of Research in Higher Education (pp. 417–452). IGI Global. https://doi.org/10.4018/979-8-3373-0654-4.ch015

Metsämuuronen, J. (2023). Seeking the real item difficulty: bias-corrected item difficulty and some consequences in Rasch and IRT modeling. Behaviormetrika, 50(1), 121–154.

Müller, M. (2020a). Item fit statistics for Rasch analysis: can we trust them? Journal of Statistical Distributions and Applications, 7(1), 5.

O’Leary, M. (2022). Linking research and teaching to benefit student learning in higher education: complexity, collaboration and a tailored approach. In International Encyclopedia of Education: Fourth Edition (pp. 258–267). Elsevier. https://doi.org/10.1016/B978-0-12-818630-5.02028-5

Prasetya, W. A., & Pratama, A. T. (2023a). Item quality analysis using the Rasch model to measure critical thinking ability in the material of the human digestive system of Biology subject in high school. Jurnal Penelitian Dan Evaluasi Pendidikan, 27(1), 76–91.

Prosekov, A. Y., Morozova, I. S., & Filatova, E. V. (2020). A Case Study of Developing Research Competency in University Students. European Journal of Contemporary Education, 9(3), 592–602.

Qiu, C., Feng, X., Reinhardt, J. D., & Li, J. (2019). Development and psychometric testing of the Research Competency Scale for Nursing Students: An instrument design study. Nurse Education Today, 79, 198–203. https://doi.org/https://doi.org/10.1016/j.nedt.2019.05.039

Risinamhodzi, D. T., & Matobobo, C. (2021). An exploration of the efficacy of project-based assessment as a tool to promote basic research skills at undergraduate level in South African rural universities. In K. T., K. C., & P. A. (Eds.), IEEE Global Engineering Education Conference, EDUCON (Vols. 2021-April, pp. 808–813). IEEE Computer Society. https://doi.org/10.1109/EDUCON46332.2021.9454029

Sukri, A., Sakti, H. G., Rizka, M. A., Lestari, I., Lukitasari, M., & Ayu, I. W. (2023). Research Skills Profile of Prospective Science Teacher Viewed from Gender (Vol. 1). Atlantis Press SARL. https://doi.org/10.2991/978-2-38476-056-5_61

Sumintono, B., & Widhiarso, W. (2015a). Aplikasi pemodelan rasch pada assessment pendidikan (1st ed.). Trim Komunikata Publishing House.

Willison, J., & Buisman-Pijlman, F. (2016). PhD prepared: research skill development across the undergraduate years. International Journal for Researcher Development, 7(1), 63–83. https://doi.org/10.1108/ijrd-07-2015-0018

Willison, J., & O’Regan, K. (2007a). Commonly known, commonly not known, totally unknown: a framework for students becoming researchers. Higher Education Research and Development, 26(4), 393–409. https://doi.org/10.1080/07294360701658609

Zapata, D. I. C., Zambrano, R. V. G., & Piloso, P. M. R. (2024a). Pedagogical strategies for the development of research skills in higher education. Salud, Ciencia y Tecnologia - Serie de Conferencias, 3. https://doi.org/10.56294/sctconf20241003




DOI: http://dx.doi.org/10.21043/thabiea.v8i2.34359

Refbacks

  • There are currently no refbacks.