Development of an Authentic Assessment Instrument for Higher-Order Thinking Skills in the Hybrid ProBL Model
DOI:
https://doi.org/10.17977/um039v11i22026p388-403Keywords:
Project Based Learning, Design Thinking, HOTS, ProBL Hybrid, Authentic Assessment, Prospective TeachersAbstract
21st-century learning requires educators not only to master content knowledge but also to design learning processes that facilitate higher-order thinking skills (HOTS). Conventional assessment methods, particularly those emphasizing multiple-choice questions and factual recall, are considered insufficient for measuring HOTS, highlighting the need for authentic assessment instruments aligned with project- and problem-based learning. This research and development (R&D) study aims to develop and validate an authentic HOTS assessment instrument based on the C4–C6 cognitive levels of Anderson & Krathwohl's taxonomy, integrated with the Hybrid ProBL (Project–Problem Based Learning) model using a design thinking approach, and to describe the results of its initial trial. Instrument development followed the 4-D model (Define, Design, Develop, and Disseminate), encompassing needs analysis, instrument design, expert validation, and limited field testing. The instrument was validated by three experts specializing in HOTS content, instructional design, and educational evaluation. A limited trial involving university students was conducted to evaluate the instrument's reliability and practicality. The results indicated that the instrument achieved an average content validity coefficient (Aiken's V) of 0.86, meeting the established content validity criterion, and a Cronbach's Alpha coefficient of 0.89, indicating high internal consistency. The developed instrument shows potential for assessing students' HOTS achievement in contextualized learning activities through project- and problem-based learning. However, this study did not statistically examine the instrument's effectiveness in improving students' HOTS. This study contributes an authentic HOTS assessment instrument that can serve as a reference for teacher education institutions in integrating innovative learning models and authentic assessment into teacher preparation programs.
References
Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and psychological measurement, 45(1), 131-142. https://doi.org/10.1177/0013164485451012
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives: complete edition. Addison Wesley Longman, Inc.
Bell, S. (2010). Project-based learning for the 21st century: Skills for the future. The clearing house, 83(2), 39-43. https://doi.org/10.1080/00098650903505415
Bereiter, C., & Scardamalia, M. (2014). Knowledge building and knowledge creation: One concept, two hills to climb. In Knowledge creation in education (pp. 35-52). Singapore: Springer Singapore. https://doi.org/10.1007/978-981-287-047-6_3
Binkley, S., Brown, C. L., Deviche, P., Donham, R. S., Elliott, J. A., Goldman, B. D., Goodman, R. L., Marler, C. A., Moore, F. L., Moore, M. C., Reppert, S. M., Stetson, M. H., Underwood, H., Watson-Whitmyre, M., Weaver, D. R., Wilson, F. E., & Wingfield, J. C. (2012). Processing of environmental information in vertebrates. Springer Science & Business Media.
Brandt, W. C. (2024). A Review of the Literature on Intercultural Understanding. Competencies of the Future. National Center for the Improvement of Educational Assessment.
Brookhart, S. M. (2010). How to assess higher-order thinking skills in your classroom. Ascd.
Brown. T. (2009). Change by Design: How Design Thinking Transforms Organization and Inspires Innovation. Harper Collins.
de la Torre-Neches, B., Rubia-Avi, M., Aparicio-Herguedas, J. L., & Rodríguez-Medina, J. (2020). Project-based learning: an analysis of cooperation and evaluation as the axes of its dynamic. Humanities and Social Sciences Communications, 7(1), 167. https://doi.org/10.1057/s41599-020-00663-z
Dewi, L. N. G. A., Widiana, I. W., & Jayanta, I. N. L. (2024). The project-based learning assessment guide (project-based learning) is oriented towards phenomenon-based learning. Journal of Education Research and Evaluation, 8(2), 362-372. https://doi.org/10.23887/jere.v8i2.74594
Ekizer, F. N., & Yildirim, S. S. (2023). 21st Century Skills and Learning Environments: ELT Students' Perceptions. Educational Research and Reviews, 18(6), 114-128.
Evenddy, S. S., Gailea, N., & Syafrizal, S. (2023). Exploring the benefits and challenges of project-based learning in higher education. PPSDP International Journal of Education, 2(2), 458-469. https://doi.org/10.59175/pijed.v2i2.148
Gulikers, J. T., Bastiaens, T. J., & Kirschner, P. A. (2004). A five-dimensional framework for authentic assessment. Educational technology research and development, 52(3), 67-86. https://doi.org/10.1007/BF02504676
Hague, C. (2024). Fostering higher order thinking skills online in higher education: A scoping review. OECD Education Working Papers. https://doi.org/10.1787/84f7756a-en.
Hidayati, D. (2024). Integrating Critical Thinking in Project-Based Learning for ESP Classes: An Indonesian Perspective. Paradigma: Jurnal Filsafat, Sains, Teknologi, dan Sosial Budaya, 30(5), 91-96. https://doi.org/10.33503/paradigma.v30i5.939
Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn?. Educational psychology review, 16(3), 235-266. https://doi.org/10.1023/B:EDPR.0000034022.16470.f3
Jia, L., Jalaludin, N. A., & Rasul, M. S. (2023). Design thinking and project-based learning (DT-PBL): A review of the literature. International Journal of Learning, Teaching and Educational Research, 22(8), 376-390. https://doi.org/10.26803/ijlter.22.8.20
Laur, D. (2013). Authentic learning experiences: A real-world approach to project-based learning. Routledge.
Liedtka, J. (2015). Perspective: Linking design thinking with innovation outcomes through cognitive bias reduction. Journal of product innovation management, 32(6), 925-938. https://doi.org/10.1111/jpim.12163
Luchang, A. L., & Nasri, N. M. (2023). Project-Based Learning (Pbl) In Enhancing Students’ Higher-Order Thinking Skills (Hots): Systematic Literature Review. International Journal of Academic Research in Progressive Education and Development, 12(4), 1651-1679. https://doi.org/10.6007/IJARPED/v12-i4/20404
Maspiroh, I., Prasetyo, Z. K., & Hermanto, H. (2025). Higher Order Thinking Skills (HOTS) of Elementary School Teacher Education Students in Indonesia: A Systematic Literature Review. In The Asian Conference on Education 2024: Official Conference Proceedings (pp. 99-110).
Miles, M. B., Huberman, A. M., & Saldaña, J. (2014). Qualitative data analysis: A methods sourcebook (3rd ed.). Thousand Oaks, CA: SAGE Publications.
OECD. (2018). A brave new world: Technology and education. OECD Publishing. https://doi.org/10.1787/9b181d3c-en
Ogunleye, B., Zakariyyah, K. I., Ajao, O., Olayinka, O., & Sharma, H. (2024). Higher education assessment practice in the era of generative AI tools. Journal of Applied Learning & Teaching, 7(1), 46-56. https://doi.org/10.37074/jalt.2024.7.1.28
Oroh, E. Z., Ali, M. I., Pelenkahu, N., Usman, H., & Rorintulus, O. A. (2025). Authentic assessment in higher education to increase critical thinking and develop metacognitive awareness. Studies in English Language and Education, 12(2), 827-844. https://doi.org/10.24815/siele.v12i2.45077
Rahayu, S., & Rosawati, E. E. (2023, March). The development of higher-order thinking skills (HOTS) assessment instrument in chemistry using socioscientific issues context: A preliminary trial. In AIP Conference Proceedings (Vol. 2572, No. 1, p. 030009). AIP Publishing LLC. https://doi.org/10.1063/5.0118624
Razzouk, R., & Shute, V. (2012). What is design thinking and why is it important?. Review of educational research, 82(3), 330-348. https://doi.org/10.3102/0034654312457429
Richey, R. C., & Klein, J. D. (2013). Design and development research. In Handbook of research on educational communications and technology (pp. 141-150). New York, NY: Springer New York. https://doi.org/10.1007/978-1-4614-3185-5_12
Safitri, K. H., Sofiyah, E., Kusmana, S., & Jaja (2024). Authentic Assessment Based on Higher Order Thinking Skills in Improving Student Literacy. British Journal of Teacher Education and Pedagogy, 3(3), 172–181. https://doi.org/10.32996/bjtep.2024.3.3.14
Strobel, J., & Van Barneveld, A. (2009). When is PBL more effective? A meta-synthesis of meta-analyses comparing PBL to conventional classrooms. Interdisciplinary journal of problem-based learning, 3(1), 44-58. https://doi.org/10.7771/1541-5015.1046
Sulistiani, S., & Yusuf, M. (2022). Development of Computer Based Test Instrument for Higher Order Thinking Skills (CBT-HOTS) on Chemical Bonding Materials. Journal of Educational Chemistry (JEC), 4(1), 31-38. https://doi.org/10.21580/jec.2022.4.1.11709
Sun, H., Xie, Y., & Lavonen, J. (2022). Exploring the structure of students’ scientific higher order thinking in science education. Thinking Skills and Creativity, 43, 100999. https://doi.org/10.1016/j.tsc.2022.100999
Thiagarajan, S., Semmel, D. S., & Semmel, M. I. (1974). Instructional development for training teachers of exceptional children: A sourcebook. Indiana University.
Thomas, J. W. (2000). A review of research on project-based learning. The Autodesk Foundation
Thornhill-Miller, B., Camarda, A., Mercier, M., Burkhardt, J.-M., Morisseau, T., Bourgeois-Bougrine, S., Vinchon, F., El Hayek, S., Augereau-Landais, M., Mourey, F., Feybesse, C., Sundquist, D., & Lubart, T. (2023). Creativity, Critical Thinking, Communication, and Collaboration: Assessment, Certification, and Promotion of 21st Century Skills for the Future of Work and Education. Journal of Intelligence, 11(3), 54. https://doi.org/10.3390/jintelligence11030054
Trilling, B., & Fadel, C. (2009). 21st century skills: Learning for life in our times. John Wiley & Sons.
UNESCO. (2015). Education 2030: Incheon Declaration and Framework for Action: Towards inclusive and equitable quality education and lifelong learning for all. https://unesdoc.unesco.org/ark:/48223/pf0000245656
Voogt, J., & Roblin, N. P. (2012). A comparative analysis of international frameworks for 21st century competences: Implications for national curriculum policies. Journal of curriculum studies, 44(3), 299-321. https://doi.org/10.1080/00220272.2012.668938
Wang, Z., Abdullah, Z., & Hu, W. (2025). A systematic review of the impact of social media on project-based learning. Sustainability, 17(8), 3680. https://doi.org/10.3390/su17083680
Wiggins, G. (1998). Educative Assessment. Designing Assessments To Inform and Improve Student Performance. Jossey-Bass Publishers.
World Economic Forum. (2020). The future of jobs report 2020. https://www.weforum.org/publications/the-future-of-jobs-report-2020/
Yanti, F. A., & Thohir, M. A. (2024). Higher order thinking skills in science learning: a systematic review from 2014-2023. International Journal of Evaluation and Research in Education, 13(4), 2419-2427. https://doi.org/10.11591/ijere.v13i4.28082
Yudha, R. P. (2023). Higher order thinking skills (HOTS) test instrument: Validity and reliability analysis with the rasch model. EduMa: Mathematics education learning and teaching, 12(1), 21-38. https://doi.org/10.24235/eduma.v12i1.9468
Zamecnik, A., Kovanović, V., Joksimović, S., Grossmann, G., Ladjal, D., & Pardo, A. (2024). The perceptions of task cohesion in collaborative learning teams. International Journal of Computer-Supported Collaborative Learning, 19(3), 369-393. https://doi.org/10.1007/s11412-024-09424-5
Zohar, A., & Dori, Y. J. (2003). Higher order thinking skills and low-achieving students: Are they mutually exclusive?. The journal of the learning sciences, 12(2), 145-181. https://doi.org/10.1207/S15327809JLS1202_1
Downloads
Published
How to Cite
Issue
Section
