Analysis of Differential Item Functioning on Students’ Mathematical Problem-Solving Ability Assessment Instruments
DOI:
https://doi.org/10.22437/edumatica.v15i2.44361Keywords:
bias assessment, differential item functioning, Mantel-Haenszel method, problem-solving ability, validity and reliability estimationAbstract
The current problem-solving ability assessment tool lacks specified criteria for fair instrument items to measure students’ abilities across genders and grade levels. To ensure score validity, unfair or biased assessment items should be removed from the test instrument. This study analyzed the quality of assessment items for mathematical problem-solving abilities, considering variables of gender and class. This research is an instrument development study with a quantitative descriptive approach. Data collection using an online survey obtained responses from 362 high school students to 10 instrument items. The data analysis technique utilized classical test theory, employing an item functioning analysis approach through the Mantel-Haenszel method. The item validity analysis indicated that not all items were classified as biased among gender groups. Two items were identified as biased according to grade level groups. The reliability estimate was 0.829, indicating a high level of consistency in the results obtained using different samples. The study’s findings validate the bias item analysis approach and provide a reliable assessment tool for evaluating high school students’ mathematics problem-solving abilities. This study also suggests using modern theoretical analysis to identify item features that distinguish students’ ability, guessing levels, and carelessness in test item responses.
Downloads
References
Abdellahi, E. A. S., Hadri, Z. E., & Iaousse, M. (2023). A New Algorithm to Compute the Correlation Matrix Implied By a Confirmatory Factor Analysis Model. 2023 3rd International Conference on Innovative Research in Applied Science, Engineering and Technology (IRASET), 1–5. https://doi.org/10.1109/IRASET57153.2023.10152922
Adams, R. J. (2005). Reliability as a Measurement Design Effect. Studies in Educational Evaluation, 31(2–3), 162–172. https://doi.org/10.1016/j.stueduc.2005.05.008
Agarwal, N. K. (2011). Verifying Survey Items for Construct Validity: A Two-Stage Sorting Procedure for Questionnaire Design in Information Behavior Research. Proceedings of the American Society for Information Science and Technology, 48(1), 1–8. https://doi.org/10.1002/meet.2011.14504801166
Agustinsa, R., Anjasari, V., & Yensy, N. A. (2023). The Effect of Problem-Based Learning Models Using Contextual Worksheets on Middle School Students’ Mathematical Problem Solving Ability. Edumatica : Jurnal Pendidikan Matematika, 13(1), 48–56. https://doi.org/10.22437/edumatica.v13i01.24387
Aiken, Lewis R. (1985). Three Coefficients for Analyzing the Reliability and Validity of Ratings. Educational and Psychological Measurement, 45(1), 131–142. https://doi.org/10.1177/0013164485451012
Akbari, A., & Shahrokhi, M. (2024). Unveiling Fairness in Scoring: A Thorough Method for Precise Cutoff Score Calculation in Education Assessment. Quality Assurance in Education, 32(3), 493–509. https://doi.org/10.1108/QAE-12-2023-0208
Algina, J., & Swaminathan, H. (2015). Psychometrics: Classical Test Theory. In J. D. B. T.-I. E. of the S. & B. S. (Second E. Wright (Ed.), International Encyclopedia of the Social & Behavioral Sciences (Second Edition) (pp. 423–430). Elsevier. https://doi.org/10.1016/B978-0-08-097086-8.42070-2
Allen, M. J., & Yen, W. M. (1979). Introduction to Measurement Theory. Brooks/Cole.
Almanasreh, E., Moles, R., & Chen, T. F. (2019). Evaluation of Methods Used for Estimating Content Validity. Research in Social & Administrative Pharmacy : RSAP, 15(2), 214–221. https://doi.org/10.1016/j.sapharm.2018.03.066
Anderson, D., Joshua, D. K., & Tindal, G. (2017). Exploring the Robustness of a Unidimensional Item Response Theory Model With Empirically Multidimensional Data. Applied Measurement in Education, 30(3), 163–177. https://doi.org/10.1080/08957347.2017.1316277
Andrade, Chittaranjan. (2020). The Inconvenient Truth About Convenience and Purposive Samples. Indian Journal of Psychological Medicine, 43(1), 86–88. https://doi.org/10.1177/0253717620977000
Annisavitri, R., Sa’dijah, C., Qohar, A., Sa’diyah, M., & Anwar, L. (2020). Analysis of Mathematical Literacy Test as a Problem-Solving Ability Assessment of Junior High School Students. AIP Conference Proceedings, 2215(1), 60002. https://doi.org/10.1063/5.0000648
Apriani, I. F., Turmudi, T., Jupri, A., & Syaodih, E. (2021). How is the Mathematical Problem-Solving Ability of Elementary School Pre-Service Teachers? Journal of Engineering Science and Technology, 16, 57–64.
Astuti, A. P., Aziz, A., Sumarti, S. S., & Bharati, D. A. L. (2019). Preparing 21st Century Teachers: Implementation of 4C Character’s Pre-Service Teacher through Teaching Practice. Journal of Physics: Conference Series, 1233(1), 12109. https://doi.org/10.1088/1742-6596/1233/1/012109
Astuti, A. T., Supahar, Mundilarto, & Istiyono, E. (2020). Development of Assessment Instruments to Measure Problem Solving Skills in Senior High School. Journal of Physics: Conference Series, 1440(1), 012063. https://doi.org/10.1088/1742-6596/1440/1/012063
Bahar, A., & Maker, C. J. (2015). Cognitive Backgrounds of Problem Solving: A Comparison of Open-ended vs. Closed Mathematics Problems. Eurasia Journal of Mathematics, Science and Technology Education, 11(6), 1531–1546. https://doi.org/10.12973/eurasia.2015.1410a
Bambang, S. R. M., Salasi, R., Hasbi, M., & Mardhiah, M. Z. (2018). The Development of an Instrument to Explore Non-Routine Problem Solving Strategies Among Mathematics Education Students. Journal of Physics: Conference Series, 1088(1), 12059. https://doi.org/10.1088/1742-6596/1088/1/012059
Barana, A., Boetti, G., & Marchisio, M. (2022). Self-Assessment in the Development of Mathematical Problem-Solving Skills. Education Sciences, 12(2), 81. https://doi.org/10.3390/educsci12020081
Blankson, A. N. (2020). Reliability, Issues of. In The Wiley Encyclopedia of Personality and Individual Differences (pp. 165–168). https://doi.org/10.1002/9781119547167.ch98
Bodoff, D. (2008). Test Theory for Evaluating Reliability of IR Test Collections. Information Processing & Management, 44(3), 1117–1145. https://doi.org/10.1016/j.ipm.2007.11.006
Brown, J. D. (2013). Classical test theory. In G. Fulcher & F. Davidson (Eds.), The Routledge Handbook of Language Testing (pp. 323–335). Routledge Taylor & Francis Group. https://doi.org/10.4324/9780203181287-35
Campbell, J. I. D. (2004). Handbook of Mathematical Cognition. Psychology Press.
Chan, David W. (2009). Developing the Impossible Figures Task to Assess Visual-Spatial Talents Among Chinese Students: A Rasch Measurement Model Analysis. Gifted Child Quarterly, 54(1), 59–71. https://doi.org/10.1177/0016986209352685
Choi, E., & Jeon, J. (2022). Factors Influencing Problem-Solving Competence of Nursing Students: A Cross-Sectional Study. Healthcare (Basel, Switzerland), 10(7), 1184. https://doi.org/10.3390/healthcare10071184
Coulacoglou, C., & Saklofske, D. H. (2017). Psychometrics and Psychological Assessment: Principles and Applications. Elsevier Academic Press.
Darma, I. K., Candiasa, I. M., Sadia, I. W., & Dantes, N. (2018). The Effect of Problem-Based Learning Model and Authentic Assessment on Mathematical Problem Solving Ability by Using Numeric Ability as the Covariable. Journal of Physics: Conference Series, 1040(1), 012035. https://doi.org/10.1088/1742-6596/1040/1/012035
DeVellis, R. F. (2006). Classical Test Theory. Medical Care, 44(11), S50–S59. https://doi.org/10.1097/01.mlr.0000245426.10853.30
Dickinson, J. R. (2015). An Empirical Comparison Of Measures Of Multiple-Choice Question Item Difficulty. In L. Robinson (Ed.), Marketing Dynamism & Sustainability: Things Change, Things Stay the Same…. Developments in Marketing Science: Proceedings of the Academy of Marketing Science (pp. 327–328). Springer International Publishing. https://doi.org/10.1007/978-3-319-10912-1_110
Douglas-Morris, J., Ritchie, H., Willis, C., & Reed, D. (2021). Identification-Based Multiple-Choice Assessments in Anatomy can be as Reliable and Challenging as Their Free-Response Equivalents. Anatomical Sciences Education, 14(3), 287–295. https://doi.org/10.1002/ase.2068
Fahrudin, D., Mardiyana, & Pramudya, I. (2019). The Analysis of Mathematic Problem Solving Ability by Polya Steps on Material Trigonometric Reviewed from Self-Regulated Learning. Journal of Physics: Conference Series, 1254(1), 012076. https://doi.org/10.1088/1742-6596/1254/1/012076
Fernandes, H. J. X. (1984). Testing and Measurement. National Education Planning, Evaluation, and Curriculum Development.
Firda, N., Suryadi, D., & Dahlan, J. A. (2023). Mathematical Problem-Solving Ability of Junior High School Students Based on Polya. Edumatica : Jurnal Pendidikan Matematika, 13(3), 273–284. https://doi.org/10.22437/edumatica.v13i03.29287
Firdausi, & Supinah, R. (2021). Development of Authentic Assessment to Improve Students’ Mathematical Problem Solving Ability. Journal of Physics: Conference Series, 1836(1), 12065. https://doi.org/10.1088/1742-6596/1836/1/012065
Fitriana, D. A., & Supahar, S. (2019). Developing an Assessment Instrument of Mathematical Problem-Solving Skills in Senior High School. International Journal of Trends in Mathematics Education Research, 2(3), 138–141. https://doi.org/10.33122/ijtmer.v2i3.81
Gamerman, D., Gonçalves, F. B., & Soares, T. M. (2018). Differential Item Functioning. In W. J. van der Linden (Ed.), Handbook of Item Response Theory: Three Volume Set (pp. 67–86). Chapman and Hall/CRC. https://doi.org/10.1201/9781315119144-57
Gozali, I., Syamsuri, S., Nindiasari, H., & Fatah, A. (2022). The Effect of Problem Based Learning on Mathematical Disposition and Students’ Problem-Solving Ability. Edumatica : Jurnal Pendidikan Matematika, 12(2), 102–110. https://doi.org/10.22437/edumatica.v12i02.15772
Grabowski, K. C., & Lin, R. (2019). Multivariate Generalizability Theory In Language Assessment. In V. Aryadoust & M. Raquel (Eds.), Quantitative Data Analysis for Language Assessment Volume I: Fundamental Techniques (pp. 54–80). https://doi.org/10.4324/9781315187815-4
Graham, J. M., Abbie, C. G., & Thompson, B. (2003). Consequences of Not Interpreting Structure Coefficients in Published CFA Research: A Reminder. Structural Equation Modeling: A Multidisciplinary Journal, 10(1), 142–153. https://doi.org/10.1207/S15328007SEM1001_7
Greeno, C. (2003). Measurement, or How Do We Know What We Know? Topic One: Validity. Family Process, 42(3), 433–435. https://doi.org/10.1111/j.1545-5300.2003.00433.x
Guarte, J. M., & Barrios, E. B. (2006). Estimation Under Purposive Sampling. Communications in Statistics - Simulation and Computation, 35(2), 277–284. https://doi.org/10.1080/03610910600591610
Guo, Wenjing, & Wind, Stefanie A. (2021). An Iterative Parametric Bootstrap Approach to Evaluating Rater Fit. Applied Psychological Measurement, 45(5), 315–330. https://doi.org/10.1177/01466216211013105
Hasanah, A. N., Priatna, N., & Yulianti, K. (2021). The Ability of Mathematical Problem Solving of Junior High School Students in Situation based Learning and Discovery Learning. Journal of Physics: Conference Series, 1806(1), 012070. https://doi.org/10.1088/1742-6596/1806/1/012070
Hattie, J. (2015). Dimensionality of Tests: Methodology. In J. D. B. T.-I. E. of the S. & B. S. (Second E. Wright (Ed.), International Encyclopedia of the Social & Behavioral Sciences: Second Edition (pp. 437–439). Elsevier. https://doi.org/10.1016/B978-0-08-097086-8.44016-X
Hidalgo, M. D., & Gómez-Benito, J. (2010). Differential Item Functioning. In P. Peterson, E. Baker, & B. B. T.-I. E. of E. (Third E. McGaw (Eds.), International Encyclopedia of Education, Third Edition (pp. 36–44). Elsevier. https://doi.org/10.1016/B978-0-08-044894-7.00242-6
Holland, P. W., & Thayer, D. T. (1988). Differential Item Performance and the Mantel-Haenszel Procedure. In Test validity (pp. 129–145). Lawrence Erlbaum Associates, Inc. https://doi.org/10.1037/14047-004
Hwang, G.-J., Hung, C.-M., & Chen, N.-S. (2014). Improving Learning Achievements, Motivations and Problem-Solving Skills Through a Peer Assessment-based Game Development Approach. Educational Technology Research and Development, 62(2), 129–145. https://doi.org/10.1007/s11423-013-9320-7
Ismail, A., Razali, S. S., Hashim, S., Abiddin, N. Z., Masek, A., & Abd Samad, N. (2021). The Integration of Problem Based Learning in Generating 21st Century Skills. 2021 IEEE 12th Control and System Graduate Research Colloquium (ICSGRC), 19–23. https://doi.org/10.1109/ICSGRC53186.2021.9515211
Istiqomah, F., Mardiyana, & Pramudya, I. (2019). Analysis of Student’s Problem Solving Ability at Junior High School. Journal of Physics: Conference Series, 1211(1), 012085. https://doi.org/10.1088/1742-6596/1211/1/012085
Jatiningtyas, P. D., Kartono, & Mindyarto, B. N. (2022). Test Instruments to Measure Non-Routine Mathematics Problem Solving Ability Grade IV Elementary School Students. Journal of Education Research and Evaluation, 6(3), 407–414. https://doi.org/10.23887/jere.v6i3.48656
Kadir, K. (2023). Students’ Mathematics Achievement Based on Performance Assessment through Problem Solving-Posing and Metacognition Level. Mathematics Teaching Research Journal, 15(3), 109–135.
Kania, N., Kusumah, Y. S., Dahlan, J. A., Nurlaelah, E., Gürbüz, F., & Bonyah, E. (2024). Constructing and Providing Content Validity Evidence through the Aiken’s Vindex Based on the Experts’ Judgments of the Instrument to Measure Mathematical Problem-Solving Skills. REID (Research and Evaluation in Education), 10(1), 64–79. https://doi.org/10.21831/reid.v10i1.71032
Kaul, R., Kaul, R., & Paul, P. (2025). Reliability study. In A. E. M. Eltorai, J. A. Bakal, & C. M. B. T.-T. C. Gibson (Eds.), Handbook for Designing and Conducting Clinical and Translational Research (pp. 247–249). Academic Press. https://doi.org/10.1016/B978-0-323-91790-2.00040-X
Kharisudin, I., & Cahyati, N. E. (2020). Problem-Solving Ability Using Mathematical Modeling Strategy on Model Eliciting Activities Based on Mathematics Self-Concept. Journal of Physics: Conference Series, 1567(3), 32067. https://doi.org/10.1088/1742-6596/1567/3/032067
Kusuma, A. B., & Untarti, R. (2020). The Mathematical Problem-Solving In Algorithm Subject. International Journal of Scientific and Technology Research, 9(4), 2982–2986.
Lewis, T. F. (2017). Evidence Regarding the Internal Structure: Confirmatory Factor Analysis. Measurement and Evaluation in Counseling and Development, 50(4), 239–247. https://doi.org/10.1080/07481756.2017.1336929
Lim Li Gek, P. (2020). Teaching through Problem Solving. In Mathematics Teaching in Singapore (pp. 175–186). WORLD SCIENTIFIC. https://doi.org/doi:10.1142/9789811220159_0011
Linn, R. L. (2010). Validity. In P. Peterson, E. Baker, & B. B. T.-I. E. of E. (Third E. McGaw (Eds.), International Encyclopedia of Education, Third Edition (pp. 181–185). Elsevier. https://doi.org/10.1016/B978-0-08-044894-7.00893-9
Magis, D., Béland, S., Tuerlinckx, F., & de Boeck, P. (2010). A General Framework and an R Package for the Detection of Dichotomous Differential Item Functioning. Behavior Research Methods, 42(3), 847–862. https://doi.org/10.3758/BRM.42.3.847
Mansyur, M. Z., & Sunendar, A. (2020). Improving Students’ Mathematical Problem Solving Ability through Metacognitive Guidance Approach. Edumatica : Jurnal Pendidikan Matematika, 10(2), 19–27. https://doi.org/10.22437/edumatica.v10i2.10494
Meguellati, S., Samia, A., Ferhat, A., Djelloul, A., & Khalifa, Z. A. (2024). A Critical Analysis of the Use of Classical Test Theory (CTT) in Psychological Testing: A Comparison with Item Response Theory (IRT). Pakistan Journal of Life and Social Sciences, 22(2), 9442–9449. https://doi.org/10.57239/PJLSS-2024-22.2.00715
Mellenbergh, Gideon J. (1982). Contingency Table Models for Assessing Item Bias. Journal of Educational Statistics, 7(2), 105–118. https://doi.org/10.3102/10769986007002105
Memon, M. A., Thurasamy, R., Ting, H., & Cheah, J.-H. (2025). Purposive Sampling: A Review and Guidelines for Quantitative Research. Journal of Applied Structural Equation Modeling, 9(1), 1–23. https://doi.org/10.47263/JASEM.9(1)01
Merino-Soto, C. (2023). Aiken’s V Coefficient: Differences in Content Validity Judgments. MHSalud, 20(1), 23–32. https://doi.org/10.15359/mhs.20-1.3
Metsämuuronen, J. (2023). Seeking the Real Item Difficulty: Bias-Corrected Item Difficulty and Some Consequences in Rasch and IRT Modeling. Behaviormetrika, 50(1), 121–154. https://doi.org/10.1007/s41237-022-00169-9
Miyata, K., Kondo, Y., Bando, K., Hara, T., & Takahashi, Y. (2024). Structural Validity of the Mini-Balance Evaluation Systems Test in Individuals With Spinocerebellar Ataxia: A Rasch Analysis Study. Archives of Physical Medicine and Rehabilitation, 105(4), 742–749. https://doi.org/10.1016/j.apmr.2023.12.015
Mueller, R. O., & Hancock, G. R. (2015). Factor Analysis and Latent Structure Analysis: Confirmatory Factor Analysis. In J. D. B. T.-I. E. of the S. & B. S. (Second E. Wright (Ed.), International Encyclopedia of the Social & Behavioral Sciences: Second Edition (pp. 686–690). Elsevier. https://doi.org/10.1016/B978-0-08-097086-8.25009-5
Muhtarom, Zuhri, M. S., & Herlambang, B. A. (2024). Analysis of Mathematics Problem-Solving Ability of Junior High School Students in Terms of Learning Independence. AIP Conference Proceedings, 3046(1), 020013. https://doi.org/10.1063/5.0194547
Muliana, N., Pada, A. U. T., & Nurmaliah, C. (2020). Content Validity of Conation Assessment. Journal of Physics: Conference Series, 1460(1), 12057. https://doi.org/10.1088/1742-6596/1460/1/012057
Mustofa, B., Mardiyana, & Slamet, I. (2020). An Analysis of Problem Solving Ability in Linear Equation Systems with Two Variables. Journal of Physics: Conference Series, 1538(1), 12099. https://doi.org/10.1088/1742-6596/1538/1/012099
Nasution, M. L., Yerizon, Y., & Gusmiyanti, R. (2018). Students’ Mathematical Problem-Solving Abilities Through the Application of Learning Models Problem Based Learning. In Ramli, Yulkifli, Festiyed, A. M., S. R., Alizar, P. D.H., & P. A. (Eds.), IOP Conference Series: Materials Science and Engineering (Vol. 335, Issue 1). Institute of Physics Publishing. https://doi.org/10.1088/1757-899X/335/1/012117
Ningsih, S., & Hidayati, K. (2022). The Role of Abstraction Ability in Mathematical Problem Solving. AIP Conference Proceedings, 2575(1), 050021. https://doi.org/10.1063/5.0109166
Nordin, M. R., Jamal, S. A., & Anuar, N. A. M. (2022). Content Validity of Research Instruments: Assessing Domestic Ecotourism in Protected Areas. Enlightening Tourism, 12(2), 565–599. https://doi.org/10.33776/et.v12i2.7123
Ouzouni, C., & Nakakis, K. (2011). Validity and Reliability of Measurement Instruments in Quantitative Studies. Nosileftiki, 50(2), 231–239.
Paek, I., & Holland, P. (2015). A Note on Statistical Hypothesis Testing Based on Log Transformation of the Mantel–Haenszel Common Odds Ratio for Differential Item Functioning Classification. Psychometrika, 80(2), 406–411. https://doi.org/10.1007/s11336-013-9394-5
Parandrengi, R., Anwar, L., & Sa’dijah, C. (2024). Analysis of Problem-Solving Ability of High School Students in Jambi. AIP Conference Proceedings, 3106(1), 050002. https://doi.org/10.1063/5.0215320
Permata, L. D., Kusmayadi, T. A., & Fitriana, L. (2018). Mathematical Problem Solving Skills Analysis about Word Problems of Linear Program Using IDEAL Problem Solver. Journal of Physics: Conference Series, 1108(1), 012025. https://doi.org/10.1088/1742-6596/1108/1/012025
Rakhmawati, I. A., Budiyono, & Saputro, D. R. S. (2019). An Analysis of Problem Solving Ability Among High School Students in Solving Linear Equation System Word Problems. Journal of Physics: Conference Series, 1211(1), 012098. https://doi.org/10.1088/1742-6596/1211/1/012098
Reffiane, F., Sudarmin, S., Wiyanto, W., & Saptono, S. (2021). Developing an Instrument to Assess Students’ Problem-Solving Ability on Hybrid Learning Model Using Ethno-STEM Approach through Quest Program. Pegem Journal of Education and Instruction, 11(4), 1–8. https://doi.org/10.47750/pegegog.11.04.01
Ridwan, M. R., Hadi, S., & Jailani. (2022a). Identification of Effectiveness Measurements and Bias Publication of Literature Results Study: A Cooperative Learning Models on Mathematics Learning Outcomes of Vocational School Students in Indonesia. Journal on Efficiency and Responsibility in Education and Science, 15(3), 189–200. https://doi.org/10.7160/eriesj.2022.150306
Ridwan, M. R., Hadi, S., & Jailani, J. (2022b). A Meta-Analysis Study on the Effectiveness of a Cooperative Learning Model on Vocational High School Students’ Mathematics Learning Outcomes. Participatory Educational Research, 9(4), 396–421. https://doi.org/10.17275/per.22.97.9.4
Ridwan, M. R., Hadi, S., & Jailani, J. (2023a). A Meta-Analysis of Numerical Aptitude’s Effect on Learning Outcomes and Mathematical Ability. TEM Journal, 12(1), 434–444. https://doi.org/10.18421/TEM121-53
Ridwan, M. R., Hadi, S., & Jailani, J. (2023b). Measurement of Psychometric Properties Numerical Aptitude Assessment Scale for Prospective High School Students: A Rasch Model Analysis. TEM Journal, 12(4), 2416–2429. https://doi.org/10.18421/TEM124-54
Ridwan, M. R., Istiyono, E., & Widihastuti, W. (2021). Test Items Analysis of Mathematical Problem Solving Ability using a Classical Test Theory Approach. Jurnal Pendidikan MIPA, 22(1), 98–111. https://doi.org/10.23960/jpmipa/v22i1.pp98-111
Ridwan, M. R., Retnawati, H., Hadi, S., & Jailani. (2021). The Effectiveness of Innovative Learning on Mathematical Problem-Solving Ability: A Meta-Analysis. International Journal of Research in Education and Science (IJRES), 7(3), 910–932. https://doi.org/10.46328/ijres.2287
Rindskopf, D. (2015). Reliability: Measurement. In International Encyclopedia of the Social & Behavioral Sciences: Second Edition (pp. 248–252). https://doi.org/10.1016/B978-0-08-097086-8.44050-X
Rios, J. A., Ling, G., Pugh, R., Becker, D., & Bacall, A. (2020). Identifying Critical 21st-Century Skills for Workplace Success: A Content Analysis of Job Advertisements. Educational Researcher, 49(2), 80–89. https://doi.org/10.3102/0013189X19890600
Roebianto, A., Savitri, S. I., Aulia, I., Suciyana, A., & Mubarokah, L. (2023). Content Validity: Definition and Procedure of Content Validation in Psychological Research. TPM - Testing, Psychometrics, Methodology in Applied Psychology, 30(1), 5–18. https://doi.org/10.4473/TPM30.1.1
Rogers, P. (2024). Best Practices for Your Confirmatory Factor Analysis: A JASP and Lavaan Tutorial. Behavior Research Methods, 56(7), 6634–6654. https://doi.org/10.3758/s13428-024-02375-7
Rosli, R., Goldsby, D., & Capraro, M. M. (2013). Assessing Students’ Mathematical Problem-Solving and Problem-Posing Skills. Asian Social Science, 9(16), 54–60. https://doi.org/10.5539/ass.v9n16p54
Rosyidi, A. H., Sari, Y. M., Fardah, D. K., & Masriyah, M. (2024). Designing Mathematics Problem-Solving Assessment with Geogebra Classroom: Proving the Instrument Validity. Journal of Education and Learning (EduLearn), 18(3), 1030–1038. https://doi.org/10.11591/edulearn.v18i3.21191
Runnels, J. (2013). Measuring Differential Item and Test Functioning Across Academic Disciplines. Language Testing in Asia, 3(1), 9. https://doi.org/10.1186/2229-0443-3-9
Ryan, J. M., & DeMark, S. (2012). Variation in Achievement Scores Related to Gender, Item Format, and Content Area Tested. In G. Tindal & T. M. Haladyna (Eds.), Large-scale Assessment Programs for All Students (pp. 64–82). Routledge. https://doi.org/10.4324/9781410605115-9
Sarstedt, M., & Mooi, E. (2019). A Concise Guide to Market Research: The Process, Data, and Methods Using IBM SPSS Statistics (3rd ed.). Springer-Verlag GmbH. https://doi.org/10.1007/978-3-662-56707-4
Scott, S. (2007). Team Performance and the Problem-Solving Approach. Journal of Industrial Technology, 23(4).
Sireci, S. G., & Soto, A. (2016). Validity and Accountability: Test Validation for 21st-Century Educational Assessments. In H. Braun (Ed.), Meeting the Challenges to Measurement in an Era of Accountability (pp. 149–167). Routledge. https://doi.org/10.4324/9780203781302-13
Sorby, S. A., Duffy, G., & Yoon, S. Y. (2022). Math Instrument Development for Examining the Relationship between Spatial and Mathematical Problem-Solving Skills. Education Sciences, 12(11), 828. https://doi.org/10.3390/educsci12110828
Spoto, A., Nucci, M., Prunetti, E., & Vicovaro, M. (2025). Improving Content Validity Evaluation of Assessment Instruments through Formal Content Validity Analysis. Psychological Methods, 30(2), 203–222. https://doi.org/10.1037/met0000545
Steyer, R. (2015). Classical (Psychometric) Test Theory. In J. D. B. T.-I. E. of the S. & B. S. (Second E. Wright (Ed.), International Encyclopedia of the Social & Behavioral Sciences (Second Edition) (pp. 785–791). Elsevier. https://doi.org/10.1016/B978-0-08-097086-8.44006-7
Stone, C. (2019). A Defense and Definition of Construct Validity in Psychology. Philosophy of Science, 86(5), 1250–1261. https://doi.org/10.1086/705567
Su, Y.-H., Sheu, C.-F., & Wang, W.-C. (2007). Computing Confidence Intervals of Item Fit Statistics in the Family of Rasch Models Using the Bootstrap Method. Journal of Applied Measurement, 8(2), 190–203.
Sweeney, S. M., Sinharay, S., Johnson, M. S., & Steinhauer, E. W. (2022). An Investigation 0f the Nature and Consequence of the Relationship Between IRT Difficulty and Discrimination. Educational Measurement: Issues and Practice, 41(4), 50–67. https://doi.org/10.1111/emip.12522
Takeda, R., Miyata, K., Tamura, S., Kobayashi, S., & Iwamoto, H. (2024). Item Distribution of the Berg Balance Scale in Older Adults with Hip Fracture: A Rasch Analysis. Physiotherapy Theory and Practice, 40(1), 136–143. https://doi.org/10.1080/09593985.2022.2109541
Thompson, D. R., & Senk, S. L. (2017). Examining Content Validity of Tests Using Teachers’ Reported Opportunity to Learn. Investigations in Mathematics Learning, 9(3), 148–155. https://doi.org/10.1080/19477503.2017.1310572
Ukobizaba, F., Nizeyimana, G., & Mukuka, A. (2021). Assessment Strategies for Enhancing Students’ Mathematical Problem-solving Skills: A Review of Literature. Eurasia Journal of Mathematics, Science and Technology Education, 17(3), em1945. https://doi.org/10.29333/ejmste/9728
Ulya, H., Sugiman, S., & Rosnawati, R. (2024). Design and Content Validity of Mathematics Creative Problem-Solving Ability Instrument for Junior High School Students. Eurasia Journal of Mathematics, Science and Technology Education, 20(6), em2462. https://doi.org/10.29333/ejmste/14661
Varma, S. (2006). Preliminary Item Statistics Using Point-Biserial Correlation and P-Values (1st Ed.). Educational Data Systems Inc.
Wahyuni, V., Kartono, K., & Susiloningsih, E. (2018). Development of Project Assessment Instruments to Assess Mathematical Problem Solving Skills on A Project-Based Learning. Journal of Research and Educational Research Evaluation, 7(2), 147–153. https://doi.org/10.15294/jere.v7i2.24501
Weng, L. C., Dai, Y. T., Huang, H. L., & Chen, S. L. (2007). Brief Review of Meanings, Measures, and Effects in Problem Solving. Journal of Nursing, 54(4), 83–87.
Widodo, S. A., Ibrahim, I., Hidayat, W., Maarif, S., & Sulistyowati, F. (2021). Development of Mathematical Problem Solving Tests on Geometry for Junior High School Students. Jurnal Elemen, 7(1), 221–231. https://doi.org/10.29408/jel.v7i1.2973
Williamson, K. (2018). Populations and Samples. In K. Williamson & G. Johanson (Eds.), Research Methods: Information, Systems, and Contexts: Second Edition (pp. 359–377). Chandos Publishing. https://doi.org/10.1016/B978-0-08-102220-7.00015-7
Xie, D., & Cobb, C. L. (2020). Item Analysis. In The Wiley Encyclopedia of Personality and Individual Differences (pp. 159–163). https://doi.org/10.1002/9781119547167.ch97
Yao, G., Wu, C., & Yang, C. (2007). Examining the Content Validity of the WHOQOL-BREF from Respondents’ Perspective by Quantitative Methods. Social Indicators Research, 85(3), 483–498. https://doi.org/10.1007/s11205-007-9112-8
Yasin, M., Huda, S., Putra, F. G., Syazali, M., Umam, R., & Widyawati, S. (2020). IMPROVE Learning Model and Learning Independence: Influence and Interaction on Mathematics Problem-Solving Abilities in Islamic Boarding School. Journal of Physics: Conference Series, 1467(1), 12003. https://doi.org/10.1088/1742-6596/1467/1/012003
Yüksel, S., Demir, P., & Alkan, A. (2019). Factors Causing Occurrence of Artificial Dif: A Simulation Study for Dichotomous Data. Communications in Statistics - Simulation and Computation, 48(7), 2004–2011. https://doi.org/10.1080/03610918.2018.1429622
Yuristia, N., & Musdi, E. (2020). Analysis of Early Mathematical Problem-Solving Ability in Mathematics Learning for Junior High School Student. Journal of Physics: Conference Series, 1554(1), 12026. https://doi.org/10.1088/1742-6596/1554/1/012026
Yusoff, M. S. B. (2019). ABC of Content Validation and Content Validity Index Calculation. Education in Medicine Journal, 11(2), 49–54. https://doi.org/10.21315/eimj2019.11.2.6
Zapata-Ospina, J. P., & García-Valencia, J. (2022). Validity Based on Content: A Challenge in Health Measurement Scales. Journal of Health Psychology, 27(2), 481–493. https://doi.org/10.1177/1359105320953477
Zieky, M. (1993). Practical Questions in the Use of DIF Statistics in Test Development. In Differential item functioning (pp. 337–347). Lawrence Erlbaum Associates, Inc. https://doi.org/10.1075/z.62.13kok
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 M. Rais Ridwan, Samsul Hadi, Faihatuz Zuhairoh
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
