Early childhood preservice teachers' debugging block-based programs: An eye tracking study

Lucas Vasconcelos; Ismahan  Arslan-Ari; Fatih Ari

doi:10.37291/2717638X.20201132

Authors

Lucas Vasconcelos University of South Carolina https://orcid.org/0000-0001-9074-203X
Ismahan Arslan-Ari University of South Carolina https://orcid.org/0000-0003-3497-7058
Fatih Ari University of South Carolina https://orcid.org/0000-0001-9512-1312

DOI:

https://doi.org/10.37291/2717638X.20201132

Keywords:

Debugging, Block-based programs, Eye tracking, Cognitive strategies, Early childhood preservice teachers

Abstract

Learning computational skills such as programming and debugging is very important for K-12 students given the increasing need of workforce proficient in computing technologies. Programming is an intricate cognitive task that entails iteratively creating and revising programs to create an artifact. Central to programming is debugging, which consists of systematically identifying and fixing program errors. Given its central role, debugging should be explicitly taught to early childhood preservice teachers so they can support their future students’ learning to program and debug errors. In this study, we propose using eye-tracking data and cued retrospective reporting to assess preservice teachers’ cognitive strategies while debugging. Several eye-tracking studies have investigated learners’ debugging strategies though the literature lacks studies (a) conducted with early childhood preservice teachers and (b) that focus on block-based programming languages, such as Scratch. The present study addresses this gap in the literature. This study used mixed methods to triangulate quantitative findings from eye movement analysis and qualitative findings about employed debugging strategies into the creation of descriptive themes. Results showed that participants developed strategies such as simultaneous review of output and code, use of beacons to narrow down the area to be debugged, and eye fixation on output to form hypotheses. But most often, debugging was not informed by a hypothesis, which led to trial and error. Study limitations and directions for future research are discussed.

References

Alqadi, B. S., & Maletic, J. I. (2017). An empirical study of debugging patterns among novice programmers. In S. Edwards, M. Caspersen, T. Barnes, & D. Garcia (Eds.), Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education—SIGCSE’17 (pp. 15–20). New York, NY: ACM Press. https://doi.org/10.1145/3017680.3017761 DOI: https://doi.org/10.1145/3017680.3017761

Aschwanden, C., & Crosby, M. (2006). Code scanning patterns in program comprehension. Paper presented at the 39th Hawaii International Conference on System Sciences. Retrieved from https://pdfs.semanticscholar.org/1153/854b79709f4d552959eebf9f7b2c0431f63a.pdf

Basu, S. (2016). Fostering synergistic learning of computational thinking and middle school science in computer-based intelligent learning environments (Unpublished doctoral dissertation). Vanderbilt University, Nashville, TN, USA.

Bau, D. (2015). Droplet, a block-based editor for text code. Journal of Computing Sciences in Colleges, 30(6), 138–144. Retrieved from https://dl.acm.org/doi/10.5555/2753024.2753052

Bednarik, R. (2012). Expertise-dependent visual attention strategies develop over time during debugging with multiple code representations. International Journal of Human-Computer Studies, 70(2), 143-155. https://doi.org/10.1016/j.ijhcs.2011.09.003 DOI: https://doi.org/10.1016/j.ijhcs.2011.09.003

Bednarik, R., Schulte, C., Budde, L., Heinemann, B., & Vrzakova, H. (2018). Eye-movement modeling examples in source code comprehension: A classroom study. In M. Joy, & P. Ihantola (Eds.), Proceedings of the 18th Koli Calling International Conference on Computing Education Research (pp. 1-8). New York, NY: ACM Press. https://doi.org/10.1145/3279720.3279722 DOI: https://doi.org/10.1145/3279720.3279722

Beller, M., Spruit, N., Spinellis, D., & Zaidman, A. (2018). On the dichotomy of debugging behavior among programmers. In P. Lago, & M. Young (Eds.), Proceedings of the 40th International Conference on Software Engineering - ICSE ’18 (pp. 1–12). New York, NY: ACM Press. https://doi.org/10.1145/3180155.3180175 DOI: https://doi.org/10.1145/3180155.3180175

Braught, G., Wahls, T., & Marlin Eby, L. (2011). The case for pair programming in the computer science classroom. ACM Transactions on Computing Education, 11(1), 1–21. https://doi.org/10.1145/1921607.1921609 DOI: https://doi.org/10.1145/1921607.1921609

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa DOI: https://doi.org/10.1191/1478088706qp063oa

Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. Paper presented at the 2012 Annual Meeting of the American Educational Research Association. Retrieved from http://scratched.gse.harvard.edu/ct/files/AERA2012.pdf

Brooks, R. E. (1983). Towards a theory of the comprehension of computer programs. International Journal of Man-Machine Studies, 18(6), 543-554. https://doi.org/10.1016/S0020-7373(83)80031-5 DOI: https://doi.org/10.1016/S0020-7373(83)80031-5

Burke, Q. (2012). The markings of a new pencil: Introducing programming-as-writing in the middle school classroom. Journal of Media Literacy Education, 4(2), 121–135.

Busjahn, T., Schulte, C., & Busjahn, A. (2011). Analysis of code reading to gain more insight in program comprehension. In A. Korhonen, & R. McCartney (Eds.), Proceedings of the 11th ACM Koli Calling International Conference on Computing Education Research (pp. 1-9). New York, NY: ACM Press. https://doi.org/10.1145/2094131.2094133 DOI: https://doi.org/10.1145/2094131.2094133

Code.org. (n.d.). CS fundamentals unplugged. Retrieved from https://code.org/curriculum/unplugged

Computer Science Teachers Association & International Society for Technology in Education (2011). Computational thinking for all (2nd ed.). Retrieved from https://www.iste.org/explore/articleDetail?articleid=152&category=Solutions&article=Computational-thinking-for-all

Cooper, S., Dann, W., & Pausch, R. (2000). Developing algorithmic thinking with Alice. In D. Colton (Ed.), The Proceedings of the Information Systems Education Conference, (pp. 506–539). Philadelphia, PA: Education Special Interest Group of the Association of Information Technology Professionals and the Foundation for Information Technology Education. Retrieved from https://pdfs.semanticscholar.org/15c0/016285c067f2f8dbbf2d7d59f9ffceb9237f.pdf

Crosby, M., Scholtz, J., & Wiedenbeck, S. (2002). The roles beacons play in comprehension for novice and expert programmers. In J. Kuljis, L. Baldwin, & R. Scoble (Eds.), Proceedings of the 14th Annual Workshop on Psychology of Programming Interest Group (pp. 58-73). London, UK: Brunel University.

Fitzgerald, S., Lewandowski, G., McCauley, R., Murphy, L., Simon, B., Thomas, L., & Zander, C. (2008). Debugging: Finding, fixing, and flailing, a multi-institutional study of novice debuggers. Computer Science Education, 18(2), 93-116. https://doi.org/10.1080/08993400802114508 DOI: https://doi.org/10.1080/08993400802114508

Ge, X., & Land, S. M. (2004). A conceptual framework for scaffolding III-structured problem-solving processes using question prompts and peer interactions. Educational Technology Research and Development, 52(2), 5-22. https://doi.org/10.1007/bf02504836 DOI: https://doi.org/10.1007/BF02504836

Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine. https://doi.org/10.1097/00006199-196807000-00014 DOI: https://doi.org/10.1097/00006199-196807000-00014

Google (n.d.). Computational thinking for educators [online course]. Retrieved from https://computationalthinkingcourse.withgoogle.com/course?use_last_location=true

Google, & Gallup. (2015). Searching for computer science: Access and barriers in U.S. K-12 education. Retrieved from http://g.co/cseduresearch

Gould, J., & Drongowski, P. (1974). An exploratory study of computer program debugging. Human Factors, 16(3), 258-277. https://doi.org/10.1177%2F001872087401600308 DOI: https://doi.org/10.1177/001872087401600308

Greene, J. C. (2007). Mixed methods in social inquiry. San Francisco, CA: Jossey-Bass.

Greene, J. C. (2008). Is mixed methods social inquiry a distinctive methodology? Journal of Mixed Methods Research, 2(1), 7-22. https://doi.org/10.1177/1558689807309969 DOI: https://doi.org/10.1177/1558689807309969

Grover, S., & Pea, R. (2013). Computational thinking in K-12: A review of the state of the field. Educational Researcher, 42(1), 38–43. https://doi.org/10.3102/0013189X12463051 DOI: https://doi.org/10.3102/0013189X12463051

Hejmady, P., & Narayanan, N. H. (2012). Visual attention patterns during program debugging with an IDE. In S. N. Spencer (Ed.), Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA’12) (pp. 197-200). New York, NY: ACM Press. https://doi.org/10.1145/2168556.2168592 DOI: https://doi.org/10.1145/2168556.2168592

Isong, B. (2014). A methodology for teaching computer programming: First year students’ perspective. International Journal of Modern Education and Computer Science, 6(9), 15–21. https://doi.org/10.5815/ijmecs.2014.09.03 DOI: https://doi.org/10.5815/ijmecs.2014.09.03

Jadud, M. C. (2005). A first look at novice compilation behaviour using BlueJ. Computer Science Education, 15(1), 25-40. https://doi.org/10.1080/08993400500056530 DOI: https://doi.org/10.1080/08993400500056530

Jordan, B., & Henderson, A. (1995). Interaction analysis: Foundations and practice. The Journal of the Learning Sciences, 4(1), 39-103. https://doi.org/10.1207/s15327809jls0401_2 DOI: https://doi.org/10.1207/s15327809jls0401_2

K-12 Computer Science Framework Steering Committee (2016). K–12 computer science framework. Retrieved from http://www.k12cs.org

Kalelioğlu, F. (2015). A new way of teaching programming skills to K-12 students: Code.org. Computers in Human Behavior, 52(1), 200–210. https://doi.org/10.1016/j.chb.2015.05.047 DOI: https://doi.org/10.1016/j.chb.2015.05.047

Kazimoglu, C., Kiernan, M., Bacon, L., & MacKinnon, L. (2012). Learning programming at the computational thinking level via digital game-play. Procedia Computer Science, 9(1), 522–531. https://doi.org/10.1016/j.procs.2012.04.056 DOI: https://doi.org/10.1016/j.procs.2012.04.056

Kim, C., Yuan, J., Oh, J., Shin, M., & Hill, R. B. (2016). Productive struggle during inquiry learning. Paper presented at the European Association for Research on Learning & Instruction SIG 20 & SIG 26 Meetings, Ghent, Belgium.

Kim, C., Yuan, J., Vasconcelos, L., Shin, M., & Hill, R. B. (2018). Debugging during block-based programming. Instructional Science, 46(5), 767–787. https://doi.org/10.1007/s11251-018-9453-5 DOI: https://doi.org/10.1007/s11251-018-9453-5

Leech, N. L., & Onwuegbuzie, A. J. (2009). A typology of mixed methods research designs. Quality & Quantity, 43(2), 265–275. https://doi.org/10.1007/s11135-007-9105-3 DOI: https://doi.org/10.1007/s11135-007-9105-3

Leedy, P. D., & Ormrod, J. E. (2013). Practical research: Planning and design. Boston: Pearson.

Leonard, J., Buss, A., Gamboa, R., Mitchell, M., Fashola, O. S., Hubert, T., & Almughyirah, S. (2016). Using robotics and game design to enhance children’s self-efficacy, STEM attitudes, and computational thinking skills. Journal of Science Education and Technology, 25(6), 860–876. https://doi.org/10.1007/s10956-016-9628-2 DOI: https://doi.org/10.1007/s10956-016-9628-2

Lin, Y. T., Wu, C. C., Hou, T. Y., Lin, Y. C., Yang, F. Y., & Chang, C. H. (2015). Tracking students’ cognitive processes during program debugging — An eye-movement approach. IEEE Transactions on Education, 59(3), 175-186. https://doi.org/10.1109/TE.2015.2487341 DOI: https://doi.org/10.1109/TE.2015.2487341

Lye, S. Y., & Koh, J. H. L. (2014). Review on teaching and learning of computational thinking through programming: What is next for K-12? Computers in Human Behavior, 41(1), 51–61. https://doi.org/10.1016/j.chb.2014.09.012 DOI: https://doi.org/10.1016/j.chb.2014.09.012

McCauley, R., Fitzgerald, S., Lewandowski, G., Murphy, L., Simon, B., Thomas, L., & Zander, C. (2008). Debugging: A review of the literature from an educational perspective. Computer Science Education, 18(2), 67–92. https://doi.org/10.1080/08993400802114581 DOI: https://doi.org/10.1080/08993400802114581

Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91(2), 358– 368. https://doi.org/10.1037/0022-0663.91.2.358 DOI: https://doi.org/10.1037/0022-0663.91.2.358

National Research Council. (2011). Successful K-12 STEM education: Identifying effective approaches in science, technology, engineering, and mathematics. Washington, DC: National Academies Press.

Obaidellah, U., Al Haek, M., & Cheng, P. C. H. (2018). A survey on the usage of eye-tracking in computer programming. ACM Computing Surveys, 51(1), 1–58. https://doi.org/10.1145/3145904 DOI: https://doi.org/10.1145/3145904

Obama, B. (2016). Computer science for all. Retrieved from https://obamawhitehouse.archives.gov/blog/2016/01/30/computer-science-all

Papavlasopoulou, S., Sharma, K., Giannakos, M., & Jaccheri, L. (2017). Using eye-tracking to unveil differences between kids and teens in coding activities. In P. Blikstein, & D. Abrahamson (Eds.), Proceedings of the 2017 Conference on Interaction Design and Children (pp. 171-181). New York, NY: ACM Press. https://doi.org/10.1145/3078072.3079740 DOI: https://doi.org/10.1145/3078072.3079740

Paul, A. M. (2016). The coding revolution. Scientific American, 315(2), 42–49. https://doi.org/10.1038/scientificamerican0816-42 DOI: https://doi.org/10.1038/scientificamerican0816-42

Pietinen, S., Bednarik, R., Glotova, T., Tenhunen, V., & Tukiainen, M. (2008). A method to study attention aspects of collaboration: Eye-tracking pair programmers simultaneously. In K. Räihä, & A. T. Duchowski (Eds.), Proceedings of the 2008 Symposium on Eye Tracking Research and Applications (pp. 39-42). Savannah, GA: ACM Press. https://doi.org/10.1145/1344471.1344480 DOI: https://doi.org/10.1145/1344471.1344480

Proctor, C. (2019). Measuring the computational in computational participation: Debugging interactive stories in middle school computer science. In K. Lund, G. Niccolai, E. Lavoué, C. Hmelo-Silver, G. Gweon, & M. Baker (Eds.), A Wide Lens: Combining Embodied, Enactive, Extended, and Embedded Learning in Collaborative Settings (Vol. 1, pp. 104–111). Lyon, France: International Society of the Learning Sciences.

Project Growing Up Thinking Scientifically [GUTS] (n.d.). Retrieved from http://www.projectguts.org/resources

Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk, N., Eastmond, E., Brennan, K., … Kafai, Y. (2009). Scratch: Programming for all. Communications of the ACM, 52(11), 60–67. https://doi.org/10.1145/1592761.1592779 DOI: https://doi.org/10.1145/1592761.1592779

Saldaña, J. (2013). The coding manual for qualitative researchers. Los Angeles, CA: SAGE.

Sengupta, P., Kinnebrew, J. S., Basu, S., Biswas, G., & Clark, D. (2013). Integrating computational thinking with K-12 science education using agent-based computation: A theoretical framework. Education and Information Technologies, 18(2), 351–380. https://doi.org/10.1007/s10639-012-9240-x DOI: https://doi.org/10.1007/s10639-012-9240-x

Sharafi, Z., Soh, Z., & Guéhéneuc, Y. (2015). A systematic literature review on the usage of eye-tracking in software engineering. Information and Software Technology, 67(1), 79–107. http://dx.doi.org/10.1016/j.infsof.2015.06.008 DOI: https://doi.org/10.1016/j.infsof.2015.06.008

Shute, V. J., Sun, C., & Asbell-Clarke, J. (2017). Demystifying computational thinking. Educational Research Review, 22(1), 142–158. https://doi.org/10.1016/j.edurev.2017.09.003 DOI: https://doi.org/10.1016/j.edurev.2017.09.003

Simon, B., Bouvier, D., Chen, T., Lewandowski, G., McCartney, R., & Sanders, K. (2008). Common sense computing (episode 4): Debugging. Computer Science Education, 18(2), 117-133. https://doi.org/10.1080/08993400802114698 DOI: https://doi.org/10.1080/08993400802114698

Strauss, A., & Corbin, J. M. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory. Thousand Oaks, CA: Sage.

Sun, J. C., & Hsu, K. Y. (2019). A smart eye-tracking feedback scaffolding approach to improving students’ learning self-efficacy and performance in a C programming course. Computers in Human Behavior, 95(1), 66–72. https://doi.org/10.1016/j.chb.2019.01.036 DOI: https://doi.org/10.1016/j.chb.2019.01.036

van Gog, T., Paas, F., van Merriënboer, J. J., & Witte, P. (2005). Uncovering the problem-solving process: Cued retrospective reporting versus concurrent and retrospective reporting. Journal of Experimental Psychology: Applied, 11(4), 237-244. https://doi.org/10.1037/1076-898X.11.4.237 DOI: https://doi.org/10.1037/1076-898X.11.4.237

Vasconcelos, L., & Kim, C. (2019). Coding in scientific modeling lessons (CS-ModeL). Educational Technology Research and Development. https://doi.org/10.1007/s11423-019-09724-w DOI: https://doi.org/10.1007/s11423-019-09724-w

Vihavainen, A., Airaksinen, J., & Watson, C. (2014). A systematic review of approaches for teaching introductory programming and their influence on success. In Proceedings of the Tenth Annual Conference on International Computing Education Research - ICER ’14, (pp. 19–26). New York, NY: ACM Press. https://doi.org/10.1145/2632320.2632349 DOI: https://doi.org/10.1145/2632320.2632349

Yen, C., Wu, P., & Lin, C. (2012). Analysis of experts’ and novices’ thinking process in program debugging. In K. C. Li, F. L. Wang, K. S. Yuen, S. K. S. Cheung, & R. Kwan (Eds.), Engaging Learners Through Emerging Technologies (Vol. 302, pp. 122–134). https://doi.org/10.1007/978-3-642-31398-1_12 DOI: https://doi.org/10.1007/978-3-642-31398-1_12

Yukselturk, E., & Altiok, S. (2017). An investigation of the effects of programming with Scratch on the preservice IT teachers’ self-efficacy perceptions and attitudes towards computer programming. British Journal of Educational Technology, 48(3), 789–801. https://doi.org/10.1111/bjet.12453 DOI: https://doi.org/10.1111/bjet.12453

Yusuf, S., Kagdi, H. H., & Maletic, J. I. (2007). Assessing the comprehension of UML class diagrams via eye tracking. In Proceedings of 15th IEEE International Conference on Program Comprehension (pp. 113–122). Alberta, Canada: IEEE. https://doi.org/10.1109/ICPC.2007.10 DOI: https://doi.org/10.1109/ICPC.2007.10