Effectiveness of Artificial Intelligence on Students’ Critical Thinking Skills: A Meta-Analysis Study
DOI:
https://doi.org/10.17977/um038v9i42026p499-511Keywords:
Artificial Intelligence, Critical Thinking, Meta-analysis, Moderator Analysis, Effect SizeAbstract
Empirical evidence on the effect of artificial intelligence (AI) on students’ critical thinking skills remains fragmented and has not been systematically synthesized. This study aims to quantify the effect of AI-based interventions on critical thinking skills, test for publication bias, and examine four moderators: region, education level, treatment duration, and type of AI used. The analysis was limited to quasi-experimental studies with a control group. A meta-analysis was conducted on studies retrieved from Scopus, ScienceDirect, and ERIC, following the PRISMA guidelines. Effect sizes were calculated using Hedges’ g under a random-effects model. The results show a significant large effect with high heterogeneity, and no evidence of publication bias was found. Moderator analyses revealed significant differences in effect size across all four factors. The largest effects were observed in studies conducted in Africa, at the higher education level, with long-duration interventions, and with mixed AI use. Overall, AI-based interventions tend to improve students’ critical thinking skills. However, the magnitude of the effect depends on how the intervention is designed and implemented. Therefore, AI integration in learning should be purposefully structured according to pedagogical frameworks and supported by sufficiently long interventions. These findings strengthen the empirical evidence on the relationship between AI and critical thinking.
References
Abbas, R. M., Nser, H., & Farid, S. (2025). Exploring the Influence of AI-Generated Content on Critical Thinking Skills of University Students. Policy Research Journal, 3(5), 690–698. https://doi.org/10.5281/zenodo.15550335
Aladini, A., Ismail, S. M., Ahmad Saleem Khasawneh, M., & Shakibaei, G. (2025). Self-directed writing development across computer/AI-based tasks: Unraveling the traces on L2 writing outcomes, growth mindfulness, and grammatical knowledge. Computers in Human Behavior Reports, 17(December 2024), 100566. https://doi.org/10.1016/j.chbr.2024.100566
Alghamdi, L. H., & Alghizzi, T. M. (2025). Educators’ reflections on AI-automated feedback in higher education: a structured integrative review of potentials, pitfalls, and ethical dimensions. Frontiers in Education, 10(November), 1–17. https://doi.org/10.3389/feduc.2025.1704820
Andrian, R. D., Widianti, Y. M., & Wati, D. A. R. (2025). The Role of Artificial Intelligence in Enhancing Critical Thinking Skills of Educational Technology Students: The Moderating Influence of Digital Literacy and Usage Regulation. Journal on Smart Learning Technologies, 1(2), 109–120.
Arimbawa, G. P. A., Parwati, N. N., Agustini, K., Suartama, I. K., & Sudarma, I. K. (2025). Effectiveness Of Microlearning-Based E-Modules In Improving Critical Thinking Abilities Of Vocational High School Students. JKTP: Jurnal Kajian Teknologi Pendidikan, 8(3), 235–247. https://doi.org/10.17977/um038v8i32025p235-247
Ayanwale, M. A., & Omeh, C. B. (2026). AI-supported problem-based learning for enhancing computational thinking skills in STEM education. Computers in Human Behavior: Artificial Humans, 7(September 2025), 100263. https://doi.org/10.1016/j.chbah.2026.100263
Baroud, S., Alaoui, M. El, Idrissi, R. J., Boucetta, N., Ghariz, G., & Seghir, H. (2025). Improving biology learning in high school with ChatGPT: A case study in Morocco. Eurasia Journal of Mathematics, Science and Technology Education, 21(7). https://doi.org/10.29333/ejmste/16606
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. (2009a). Introduction to Meta-Analysis. John Wiley & Sons.
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009b). Introduction to Meta-Analysis. In A Guide to the Scientific Career: Virtues, Communication, Research, and Academic Writing. https://doi.org/10.1002/9781118907283.ch68
Brazão, P., & Tinoca, L. (2025). Artificial intelligence and critical thinking: a case study with educational chatbots. Frontiers in Education, 10(September), 1–12. https://doi.org/10.3389/feduc.2025.1630493
Bueno-Picazo, M., & Tirado-Olivares, S. (2026). Generative AI in Geography Education: Content Creation and Conversational AI-Supported Learning to Promote Environmental Awareness. European Journal of Geography, 17(2), S.34-S.48. https://doi.org/10.48088/ejg.m.bue.17.2.034.048
Cai, J., Tan, Y. H., & Lin, S. E. (2026). Rethinking literature education: A quasi-experimental study of the production-oriented approach for english majors in China. Acta Psychologica, 266(March), 106808. https://doi.org/10.1016/j.actpsy.2026.106808
Cao, Y., & Gao, Q. (2026). Pedagogically Customized Machine Translation for Cultivating Critical Thinking in Foreign Language Education. Journal of Cases on Information Technology, 28(1), 1–17. https://doi.org/10.4018/JCIT.401921
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159. https://doi.org/10.1037/0033-2909.112.1.155
Dai, Y., Lin, Z., Liu, A., Dai, D., & Wang, W. (2024). Effect of an Analogy-Based Approach of Artificial Intelligence Pedagogy in Upper Primary Schools. Journal of Educational Computing Research, 61(8), 159–186. https://doi.org/10.1177/07356331231201342
Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ, 315(7109), 629–634. https://doi.org/10.1136/bmj.315.7109.629
Eltahir, M. E., & Babiker, F. M. E. (2024). The Influence of Artificial Intelligence Tools on Student Performance in e-Learning Environments: Case Study. Electronic Journal of E-Learning, 22(9), 91–110. https://doi.org/10.34190/ejel.22.9.3639
Essel, H. B., Vlachopoulos, D., Essuman, A. B., & Amankwa, J. O. (2024). ChatGPT effects on cognitive skills of undergraduate students: Receiving instant responses from AI-based conversational large language models (LLMs). Computers and Education: Artificial Intelligence, 6(December 2023), 100198. https://doi.org/10.1016/j.caeai.2023.100198
Facione, P. A. (2011). Critical thinking: what it is and why it counts. Insight Assessment, 5(1), 1–23. https://www.researchgate.net/profile/Peter_Facione/publication/251303244_Critical_Thinking_What_It_Is_and_Why_It_Counts/links/5849b49608aed5252bcbe531/Critical-Thinking-What-It-Is-and-Why-It-Counts.pdf
Gao, J., Zhang, J., & Li, Y. (2025). Do AI chatbot-integrated writing tasks influence writing self-efficacy and critical thinking ability? An exploratory study. Computers and Education: Artificial Intelligence, 9(September 2024), 100472. https://doi.org/10.1016/j.caeai.2025.100472
García-Carmona, A. (2025). Scientific Thinking and Critical Thinking in Science Education : Two Distinct but Symbiotically Related Intellectual Processes. Science and Education, 34(1), 227–245. https://doi.org/10.1007/s11191-023-00460-5
Gerlich, M. (2025). AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking. Societies, 15(1), 1–28. https://doi.org/10.3390/soc15010006
Glass, G. V. (1976). Primary, Secondary, and Meta-Analysis of Research.
Hassen, M. (2025). The Impact of AI on Students’ Reading, Critical Thinking, and Problem-Solving Skills. American Journal of Education and Information Technology, 9(2), 82–90. https://doi.org/10.11648/j.ajeit.20250902.12
Hedges, L. V., & Olkin, I. (1985). Statistical Methods for Meta-Analysis. Academic Press.
Heydarnejad, T. (2025). Unmasking the impacts of self-evaluation in AI-supported writing instruction on EFL learners’ emotion regulation, self-competence, motivation, and writing achievement. Computers and Education: Artificial Intelligence, 9(June), 100494. https://doi.org/10.1016/j.caeai.2025.100494
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in knowledgebases. Journal of Intelligent Information Systems, 27(2), 159–184. https://doi.org/10.1007/s10844-006-2974-4
Hou, C., Zhu, G., & Sudarshan, V. (2025). The role of critical thinking on undergraduates’ reliance behaviours on generative AI in problem-solving. British Journal of Educational Technology, 56(5), 1919–1941. https://doi.org/10.1111/bjet.13613
İdil, Ş., Gülen, S., & Dönmez, İ. (2024). What Should We Understand from PISA 2022 Results? Journal of STEAM Education, 7(1), 1–9. https://doi.org/10.55290/steam.1415261
Ilgun Dibek, M., Sahin Kursad, M., & Erdogan, T. (2025). Influence of artificial intelligence tools on higher order thinking skills: a meta-analysis. Interactive Learning Environments, 33(3), 2216–2238. https://doi.org/10.1080/10494820.2024.2402028
Jose, B., Cherian, J., Verghis, A. M., Varghise, S. M., S, M., & Joseph, S. (2025). The cognitive paradox of AI in education: between enhancement and erosion. Frontiers in Psychology, 16. https://doi.org/10.3389/fpsyg.2025.1550621
Kamel, H. (2024). Understanding the impact of AI Hallucinations on the university community. Cybrarians Journal, 73, 111–134. https://doi.org/10.70000/cj.2024.73.622
Khan, I., Ahmad, A. R., Jabeur, N., & Mahdi, M. N. (2021). An artificial intelligence approach to monitor student performance and devise preventive measures. Smart Learning Environments, 8(1). https://doi.org/10.1186/s40561-021-00161-y
Kooli, C., Yusuf, N., & Sarhan, M. Y. (2026). Colloquial engagement theory with AI awareness (CET-AIA): A new creative pedagogical framework for ethical assessment in the age of artificial intelligence. Thinking Skills and Creativity, 60(November 2025), 102051. https://doi.org/10.1016/j.tsc.2025.102051
Lee, C. C., & Low, M. Y. H. (2024). Using genAI in education: the case for critical thinking. Frontiers in Artificial Intelligence, 7. https://doi.org/10.3389/frai.2024.1452131
Li, C., & Long, J. (2025). Comparing AI-Assisted and Traditional Teaching in College English: Pedagogical Benefits and Learning Behaviors. Information (Switzerland), 16(10), 1–19. https://doi.org/10.3390/info16100895
Li, T., Krajcik, J. S., & Spiro, R. (2025). The Transformative Collaboration of Human Intelligence and Artificial Intelligence in Designing Knowledge-in-Use Science Assessment for Learning. Journal of Science Education and Technology. https://doi.org/10.1007/s10956-025-10275-4
Li, Y., Kong, H., & Sciences, S. (2025). Education Studies Addressing “ Hallucinations ” in AI ‑ Generated Content : Strategies for Developing Student Fact ‑ Checking and. 1(2), 48–62.
Linsih, R. A., Soepriyanto, Y., & Praherdhiono, H. (2025). The Effect Of Collaborative Inquiry Based Learning Model And Learning Motivation On Critical Thinking Skills Of Elementary School Students. JKTP: Jurnal Kajian Teknologi Pendidikan, 8(2), 118–127. https://doi.org/10.17977/um038v8i22025p118
Liu, Q., & Tu, C. C. (2024). Improving Critical Thinking through AI-Supported Socio-Scientific Issues Instruction. Journal of Logistics, Informatics and Service Science, 11(3), 52–65. https://doi.org/10.33168/JLISS.2024.0304
Liu, Z., Gao, Y., Yang, Y., Kong, X., & Zhao, L. (2024). Exploring the influence of students’ perceptions of face-to-face collaboration on cognitive engagement and learning outcomes in collaborative programming. Acta Psychologica, 248(July), 104393. https://doi.org/10.1016/j.actpsy.2024.104393
Luan, J., Halim, H. A., & Saad, M. R. M. (2026). The Effect of AI and Multiple Intelligences in Project-Based Learning on English Achievement. International Journal of Computer-Assisted Language Learning and Teaching, 16(1), 1–28. https://doi.org/10.4018/IJCALLT.400912
Mai, H. T. (2026). Evaluating the impact of using ChatGPT on EFL students’ argumentative skills. Discover Education, 5(1), 1–20. https://doi.org/10.1007/s44217-026-01153-1
Maia, J. de S. Z., Bueno, A. P. A., & Sato, J. R. (2023). Applications of Artificial Intelligence Models in Educational Analytics and Decision Making: A Systematic Review. World, 4(2), 288–313. https://doi.org/10.3390/world4020019
Najah, E. F., Syaharuddin, Gatan, P. R., Mohammed, R. A., & Abdillah. (2024). The role of Information Literacy in Enhancing Students’ Critical Thinking Skills. Jurnal Literasi Perpustakaan Dan Informasi UHO: Jurnal Penelitian Kajian Perpustakaan Dan Informasi, 4(3), 129–139. https://doi.org/10.52423/jlpi.v4i3.15
Nusivera, E., Hikmat, A., & Ghani, A. R. A. (2025). Integration of Chat-GPT Usage in Language Learning Model to Improve Argumentation Skills, Complex Comprehension Skills, and Critical Thinking Skills. International Journal of Learning, Teaching and Educational Research, 24(2), 375–390. https://doi.org/10.26803/ijlter.24.2.19
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Bmj, 372. https://doi.org/10.1136/bmj.n71
Park, S. A., & Kim, H. Y. (2025). Development and effects of a scenario-based labor nursing simulation education program using an artificial intelligence tutor: a quasi-experimental study. Women’s Health Nursing, 31(2), 143–154. https://doi.org/10.4069/whn.2025.06.18
Partnership for 21 st Century Skills. (2015). Partnership for 21St Century Skills-Core Content Integration. Ohio Department of Education, 1. www.P21.org.
Piaget, J. (1970). Science of Education and the Psychology of the Child. Orion Press. https://books.google.co.id/books?id=ZwedAAAAMAAJ
Sinaga, A. I., Nasution, A. F., & Albina, M. (2025). Effect Of Artificial Intelligence Use On Creativity In Writing Scientific Works. JKTP: Jurnal Kajian Teknologi Pendidikan, 8(3), 226–234. https://doi.org/10.17977/um038v8i32025p226-234
Song, D., Zhang, P., Zhu, Y., Qi, S., Yang, Y., Gong, L., & Zhou, L. (2025). Effects of generative artificial intelligence on higher-order thinking skills and artificial intelligence literacy in nursing undergraduates: A quasi-experimental study. Nurse Education in Practice, 88(81), 104549. https://doi.org/10.1016/j.nepr.2025.104549
Sun, X., Liu, C., Li, P., & Cai, Z. (2026). Effects of Generative AI Tools On Students’ Critical Thinking: A Three-Level Meta-Analysis. Educational Psychology Review, 38(1), 84. https://doi.org/10.1007/s10648-026-10180-1
Sun, Y., Sheng, D., Zhou, Z., & Wu, Y. (2024). AI hallucination: towards a comprehensive classification of distorted information in artificial intelligence-generated content. Humanities and Social Sciences Communications, 11(1), 1–14. https://doi.org/10.1057/s41599-024-03811-x
Tan, X., Cheng, G., & Ling, M. H. (2025). Enhancing teachers’ AI competency: A professional development intervention study based on intelligent-TPACK framework. Computers and Education: Artificial Intelligence, 9(November), 100521. https://doi.org/10.1016/j.caeai.2025.100521
Tian, J., & Zhang, R. (2025). Learners’ AI dependence and critical thinking: The psychological mechanism of fatigue and the social buffering role of AI literacy. Acta Psychologica, 260(October), 105725. https://doi.org/10.1016/j.actpsy.2025.105725
Vygotsky, L. S. (1978). Mind in Society: Development of Higher Psychological Processes. Harvard University Press. https://books.google.co.id/books?id=RxjjUefze_oC
Wiboolyasarin, W., Wiboolyasarin, K., Suwanwihok, K., Jinowat, N., & Muenjanchoey, R. (2024). Synergizing collaborative writing and AI feedback: An investigation into enhancing L2 writing proficiency in wiki-based environments. Computers and Education: Artificial Intelligence, 6(March), 100228. https://doi.org/10.1016/j.caeai.2024.100228
Yas, N., Abdulrahim, H., Rabadi, I. G., & Abdijabar, Z. (2025). Protecting the civil rights of the author from the risks of artificial intelligence. Research Journal in Advanced Humanities, 6(1), 1–9. https://doi.org/10.58256/s7rhdm72
Zablotska, L., Chernii, L., Meleshchenko, V., Olendr, T., & Tsar, I. (2026). Developing Students’ Critical Thinking through AI-Enhanced Foreign Language Instruction. Arab World English Journal, 3, 132–146. https://doi.org/10.24093/awej/ai3.9
Zhai, C., Wibowo, S., & Li, L. D. (2024). The effects of over-reliance on AI dialogue systems on students’ cognitive abilities: a systematic review. Smart Learning Environments, 11(1). https://doi.org/10.1186/s40561-024-00316-7
Zhao, Y., Yue, Y., Sun, Z., Jiang, Q., & Li, G. (2025). Does Generative Artificial Intelligence Improve Students’ Higher-Order Thinking? A Meta-Analysis Based on 29 Experiments and Quasi-Experiments. Journal of Intelligence, 13(12), 1–21. https://doi.org/10.3390/jintelligence13120160
Zheng, K., Shen, Z., Chen, Z., Che, C., & Zhu, H. (2024). Application of AI-empowered scenario-based simulation teaching mode in cardiovascular disease education. BMC Medical Education, 24(1). https://doi.org/10.1186/s12909-024-05977-z
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Santa Aulia Devi Rachmadhani, Ali Muhson, Risky Setiawan

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Jurnal Kajian Teknologi Pendidikan allows readers to read, download, copy, distribute, print, search, or link to the full texts of its articles and allow readers to use them for any other lawful purpose. The journal allows the author(s) to hold the copyright without restrictions. Finally, the journal allows the author(s) to retain publishing rights without restrictions.
- Authors are allowed to archive their submitted articles in an open access repository.
- Authors are allowed to archive the final published article in an open access repository with an acknowledgment of its initial publication in this journal.










