YAPAY ZEKÂ DESTEKLİ DENEY TASARIM SÜRECİNDE FEN BİLİMLERİ ÖĞRETMEN ADAYLARININ TASARIMLARINA YÖNELİK CHATGPT DEĞERLENDİRMELERİNİN İNCELENMESİ

AN EXAMINATION OF CHATGPT’S EVALUATIONS OF PRE-SERVICE SCIENCE TEACHERS’ DESIGNS IN THE AI-SUPPORTED EXPERIMENT DESIGN PROCESS

Abstract

The aim of this study is to examine ChatGPT’s evaluations of pre-service science teachers’ designs within an AI-supported experiment design process. The study was conducted using a case study design, one of the qualitative research methods. The sample consisted of 32 pre-service science teachers enrolled in the Faculty of Education at a public university in Izmir during the 2024 – 2025 academic year. Data were collected on a voluntary basis during individual sessions lasting approximately one hour. In the data collection process, digitally structured forms developed by researchers using Microsoft Excel macros were utilized. Data obtained from the participants’ original experiment designs, the revisions made in line with ChatGPT feedback, and the textual records of their interactions with ChatGPT were analyzed through content analysis using MAXQDA 24 software. The coding process was reviewed based on expert feedback, and inter-researcher agreement was established. The inter-rater reliability coefficient between the two coders was determined to be 0.91. The findings of the study reveal that pre-service science teachers used ChatGPT for consulation, validation, and refinement during the experiment design process, and that they benefited from ChatGPT’s evaluations particularly in the stages of identifying variables, determining experimental procedures, and ensuring STEM integration in experiment design.

Keywords: ChatGPT, experiment design, pre-service science teachers, qualitative research, artificial intelligence tools.

Öz

Bu çalışmanın amacı yapay zekâ destekli deney tasarım sürecinde fen bilimleri öğretmen adaylarının tasarımlarına yönelik ChatGPT değerlendirmelerinin incelenmesidir. Nitel araştırma yöntemlerinden durum çalışması deseninde yürütülen çalışmanın örneklemini 2024 – 2025 eğitim-öğretim yılında İzmir’de bir devlet üniversitesinin Eğitim Fakültesinde öğrenim gören 32 fen bilimleri öğretmen adayı oluşturmaktadır. Çalışmanın verileri yaklaşık bir saat süren bireysel uygulama oturumları sırasında gönüllülük esasına dayalı olarak toplanmıştır. Veri toplama sürecinde araştırmacılar tarafından geliştirilen Microsoft Excel makroları ile yapılandırılmış dijital formlar kullanılmıştır. Öğretmen adaylarının özgün deney tasarımları, ChatGPT dönütleri doğrultusunda gerçekleştirilen düzenlemeler ve katılımcıların ChatGPT ile olan etkileşimlerine ait metinlerden elde edilen veriler MAXQDA 24 yazılımı kullanılarak içerik analiziyle çözümlenmiş, kodlama süreci uzman görüşü doğrultusunda gözden geçirilerek araştırmacılar arası uyum sağlanmıştır. İki kodlayıcı tarafından yapılan kodlamalar arasındaki puanlayıcılar arası güvenirlik katsayısı 0.91 olarak belirlenmiştir. Çalışmanın bulguları öğretmen adaylarının ChatGPT’yi deney tasarım sürecinde danışma, doğrulama ve geliştirme amacıyla kullandıklarını göstermekte ve öğretmen adaylarının deney tasarımının değişken belirleme, uygulama basamaklarını belirleme ve STEM entegrasyonu sağlama aşamalarında ChatGPT değerlendirmelerinden yararlanmakta olduklarını ortaya koymaktadır.

Anahtar Terimler: ChatGPT, deney tasarımı, fen bilimleri öğretmen adayları, nitel araştırma, yapay zekâ araçları.

KAYNAKÇA

Aguinis, H., & Lawal, S. (2012). Conducting field experiments using eLancing's natural environment. Journal of Business Venturing, 27(4), 493–505. https://doi.org/10.1016/j.jbusvent.2012.01.002

Ali, Z., Younis, S., Ahmad, N., Saba, F., & Ullah, N. (2025). Teachers’ perspective of technology integration effects on students learning at university level. Journal of Rehabilitation Research and Current Updates, 1(1), 1–6.

Alvarez, C., Alarcon, R., & Nussbaum, M. (2011). Implementing collaborative learning activities in the classroom supported by one-to-one mobile computing: A design-based process. Journal of Systems and Software, 84(11), 1961–1977. https://doi.org/10.1016/j.jss.2011.07.011

An, Q., Yang, J., Xu, X., Zhang, Y., & Zhang, H. (2024). Decoding AI ethics from users' lens in education: A systematic review. Heliyon, 10(18), https://doi.org/10.1016/j.heliyon.2024.e39357

Ateş, S. (2005). Öğretmen adaylarının değişkenleri belirleme ve kontrol etme yeteneklerinin geliştirilmesi. Gazi Üniversitesi Gazi Eğitim Fakültesi Dergisi, 25(1), 21–39.

Brotherton, P. N., & Preece, P. F. W. (1995). Science process skills: Their nature and interrelationships. Research in Science & Technological Education, 13(1), 5–12. https://doi.org/10.1080/0263514950130101

Cai, H., Han, B., Sun, J., Li, X., & Wong, L. H. (2025). Harnessing AI for teacher education to promote inclusive education: Investigating the effects of ChatGPT-supported lesson plan critiques on the development of pre-service teachers' lesson planning skills. The Internet and Higher Education, 64, https://doi.org/10.1016/j.iheduc.2025.101022

Chellappa, V., & Luximon, Y. (2024). Understanding the perception of design students towards ChatGPT. Computers and Education: Artificial Intelligence, 7, https://doi.org/10.1016/j.caeai.2024.100281

Çibik, N. F., & Boz-Yaman, B. (2025). The effect of a cross-curricular course on pre-service teachers’ sustainable development attitudes and mathematical modeling self-efficacy beliefs. International Journal of Science and Mathematics Education, 23(4), 1033–1056. https://doi.org/10.1007/s10763-024-10497-9

de Souza, B., Serrano de Andrade Neto, A., & Roazzi, A. (2024). The generative AI revolution, cognitive mediation networks theory and the emergence of a new mode of mental functioning: Introducing the Sophotechnic Mediation scale. Computers in Human Behavior: Artificial Humans, 2(1), https://doi.org/10.1016/j.chbah.2024.100042

Dobrev, D. (2005). Formal definition of artificial intelligence. International Journal "Information Theories & Applications", 12(3), 277–285.

Dockendorff, M., & Zaccarelli, F. G. (2025). Successfully preparing future mathematics teachers for digital technology integration: A literature review. International Journal of Mathematical Education in Science and Technology, 56(5), 948–979. https://doi.org/10.1080/0020739X.2024.2309273

Fijačko, N., Creber, R., Abella, B., Kocbek, P., Metličar, Š., Greif, R., & Štiglic, G. (2024). Using generative artificial intelligence in bibliometric analysis: 10 years of research trends from the European Resuscitation Congresses. Resuscitation Plus, 18, https://doi.org/10.1016/j.resplu.2024.100584

Fischer, F., Sommerhoff, D., & Keune, A. (2023). Perspectives on learning from the learning sciences. In International Encyclopedia of Education. Elsevier. https://doi.org/10.1016/B978-0-12-818630-5.14007-2

Fitzpatrick, M., & Leavy, A. (2025). Reciprocal interplays in becoming STEM learners and teachers: Preservice teachers’ evolving understandings of integrated STEM education. International Journal of Mathematical Education in Science and Technology, 1–30. https://doi.org/10.1080/0020739X.2025.2472260

Fraenkel, J. R., & Wallen, N. E. (1990). How to design and evaluate research in education. McGraw-Hill.

Galindo-Domínguez, H., Delgado, N., Campo, L., & Losada, D. (2024). Relationship between teachers’ digital competence and attitudes towards artificial intelligence in education. International Journal of Educational Research, 126, https://doi.org/10.1016/j.ijer.2024.102381

García-Carmona, A., Muñoz-Franco, G., & Cruz-Guzmán, M. (2025). Integration of engineering practices into primary science classrooms: What does educational research tell us? Science & Education, 1–30. https://doi.org/10.1007/s11191-025-00616-5

Gilbert, A., Suh, J., & Choudhry, F. (2025). Exploring the development of preservice teachers’ visions of equity through science and mathematics integration. International Journal of Science and Mathematics Education, 23(2), 489–514. https://doi.org/10.1007/s10763-024-10467-1

Hämäläinen, R., & Oksanen, K. (2012). Challenge of supporting vocational learning: Empowering collaboration in a scripted 3D game – How does teachers’ real-time orchestration make a difference? Computers & Education, 59(3), 856–869. https://doi.org/10.1016/j.compedu.2012.01.002

Hodson, D. (1988). Experiments in science and science teaching. Educational Philosophy and Theory, 20(2), 53–66. https://doi.org/10.1111/j.1469-5812.1988.tb00144.x

Hofstein, A., & Lunetta, V. N. (1982). The role of the laboratory in science teaching: Neglected aspects of research. Review of Educational Research, 52(2), 201–217. https://doi.org/10.3102/00346543052002201

Hovardas, T., Ter Vrugte, J., Zacharia, Z., & de Jong, T. (2023). Simulations and games for learning. In International Encyclopedia of Education. Elsevier. https://doi.org/10.1016/B978-0-12-818630-5.14074-6

Huang, Y., Chen, P., Lee, H., Sandnes, F., & Wu, T. (2025). ChatGPT-enhanced mobile instant messaging in online learning: Effects on student outcomes and perceptions. Computers in Human Behavior, 168, https://doi.org/10.1016/j.chb.2025.108659

Javaid, M., Haleem, A., Singh, R., Khan, S., & Khan, I. (2023). Unlocking the opportunities through ChatGPT tool towards ameliorating the education system. BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 3(3), https://doi.org/10.1016/j.tbench.2023.100115

Jayawardena, C., Gunathilake, Y., & Ihalagedara, D. (2025). Dental students’ learning experience: Artificial intelligence vs human feedback on assignments. International Dental Journal, 75(1), https://doi.org/10.1016/j.identj.2024.12.022

Jia, F., Sun, D., & Looi, C. K. (2024). Artificial intelligence in science education (2013–2023): Research trends in ten years. Journal of Science Education and Technology, 33(1), 94–117. https://doi.org/10.1007/s10956-023-10077-6

Kewalramani, S., Devi, A., & Ng, A. (2025). Supporting Early Childhood Preservice Teachers to Effectively Integrate STEM in Their Future Teaching Practice. Education Sciences, 15(2). https://doi.org/10.3390/educsci15020189

Kılıç, G. B., Yardımcı, E., & Metin, D. (2009). Fen öğretiminde değişkenler nasıl adlandırılabilir? Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 9(2), 13–26.

Kok, J. N., Boers, E. J., Kosters, W. A., Van der Putten, P., & Poel, M. (2009). Artificial intelligence: Definition, trends, techniques, and cases. In Artificial Intelligence (pp. 1-20). Eolss Publishers.

Kovari, A. (2025). A systematic review of AI-powered collaborative learning in higher education: Trends and outcomes from the last decade. Social Sciences & Humanities Open, 11, https://doi.org/10.1016/j.ssaho.2025.101335

Li, B., Lowell, V., Wang, C., & Li, X. (2024). A systematic review of the first year of publications on ChatGPT and language education: Examining research on ChatGPT’s use in language learning and teaching. Computers and Education: Artificial Intelligence, 7, https://doi.org/10.1016/j.caeai.2024.100266

Li, T., & Fu, Z. (2024). Formulating a descriptive framework and tagging system for design futures practice cases: Enabling heuristic tool development for design education and creative inspiration. Thinking Skills and Creativity, 53, https://doi.org/10.1016/j.tsc.2024.101536

Liu, J., Sun, D., Sun, J., Wang, J., & Yu, P. (2025). Designing a generative AI enabled learning environment for mathematics word problem solving in primary schools: Learning performance, attitudes and interaction. Computers and Education: Artificial Intelligence, 9, https://doi.org/10.1016/j.caeai.2025.100438

Madanchian, M., & Taherdoost, H. (2025). Decision-making criteria for AI tools in digital education. Digital Engineering, 7, https://doi.org/10.1016/j.dte.2025.100069

Martin-Villalba, C., Urquia, A., & Dormido, S. (2012). Development of virtual-labs for education in chemical process control using Modelica. Computers & Chemical Engineering, 39, 169–177. https://doi.org/10.1016/j.compchemeng.2011.10.010

Millar, R. (2004). The role of practical work in the teaching and learning of science. National Academy of Sciences.

Milli Eğitim Bakanlığı. (2024). Türkiye Yüzyılı Maarif Modeli: Fen bilimleri dersi öğretim programı (İlkokul ve ortaokul). Ankara: Milli Eğitim Bakanlığı.

Ozturk, N., Aydin-Günbatar, S., & Roehrig, G. H. (2025). Elementary science teachers’ engineering integration after long-term in-service training program with and without curriculum material support. Research in Science & Technological Education, 43(1), 170–190. https://doi.org/10.1080/02635143.2023.2260996

Playfoot, D., Quigley, M., & Thomas, A. (2024). Hey ChatGPT, give me a title for a paper about degree apathy and student use of AI for assignment writing. The Internet and Higher Education, 62, https://doi.org/10.1016/j.iheduc.2024.100950

Popenici, S. A., & Kerr, S. (2017). Exploring the impact of artificial intelligence on teaching and learning in higher education. Research and Practice in Technology Enhanced Learning, 12(1), https://doi.org/10.1186/s41039-017-0062-8

Salas-Pilco, S. Z., Xiao, K., & Hu, X. (2022). Artificial intelligence and learning analytics in teacher education: A systematic review. Education Sciences, 12(8), https://doi.org/10.3390/educsci12080569

Savin-Baden, M., & Major, C. H. (2023). Qualitative research: The essential guide to theory and practice (2nd ed.). Routledge. https://doi.org/10.4324/9781003377986

Simmons, A. B., & Chappell, S. G. (1988). Artificial intelligence: Definition and practice. IEEE Journal of Oceanic Engineering, 13(2), 14–42. https://doi.org/10.1109/48.551

Song, P., & Wang, C. (2023). Can ChatGPT replace scientists? Science Bulletin, 68(11), 1143–1146. https://doi.org/10.1016/j.scib.2023.08.032

Suleiman, A., von Wedel, D., Munoz-Acuna, R., Redaelli, S., Santarisi, A., Seibold, E., ... & Schaefer, M. (2024). Assessing ChatGPT's ability to emulate human reviewers in scientific research: A descriptive and qualitative approach. Computer Methods and Programs in Biomedicine, 254, https://doi.org/10.1016/j.cmpb.2024.108313

Wei, X., Wang, L., Koszalka, T., Lee, L., & Liu, R. (2025). Enhancing pre-service teachers' reflective thinking skills through generative AI-assisted digital storytelling creation: A three-dimensional framework analysis. Computers & Education, 235, https://doi.org/10.1016/j.compedu.2025.105356

Yun, M., & Crippen, K. J. (2025). Computational thinking integration into pre-service science teacher education: A systematic review. Journal of Science Teacher Education, 36(2), 225–254. https://doi.org/10.1080/1046560X.2024.2390758

Zeng, F., Gan, W., Wang, Y., & Yu, P. (2025). Distributed training of large language models: A survey. Natural Language Processing Journal, 12, https://doi.org/10.1016/j.nlp.2025.100174

Zhang, Z., & Huang, X. (2024). The impact of chatbots based on large language models on second language vocabulary acquisition. Heliyon, 10(2), https://doi.org/10.1016/j.heliyon.2024.e25370

Zhu, S., Li, Q., Yao, Y., Li, J., & Zhu, X. (2025). Improving writing feedback quality and self-efficacy of pre-service teachers in Gen-AI contexts: An experimental mixed-method design. Assessing Writing, 66, https://doi.org/10.1016/j.asw.2025.100960

Zou, Y., Kuek, F., Feng, W., & Cheng, X. (2025). Digital learning in the 21st century: Trends, challenges, and innovations in technology integration. In Frontiers in Education, 10, Frontiers Media SA. https://doi.org/10.3389/feduc.2025.1562391