Category: Professional Learning Facilitator

Provides information about professional learning and the advancement of teaching using technology.
– Designing based on needs assessment
– Facilitating active learning
– Evaluating the impact of professional learning

Posted on

Potential of LLMs and Automated Text Analysis in Interpreting Student Course Feedback

Integrating Large Language Models (LLMs) with automated text analysis tools offers a novel approach to interpreting student course feedback. As educators and administrators strive to refine teaching methods and enhance learning experiences, leveraging AI’s capabilities could unlock more profound insights from student feedback. Traditionally seen as a vast collection of qualitative data filled with sentiments, preferences, and suggestions, this feedback can now be more effectively analyzed. This blog will explore how LLMs can be utilized to interpret and classify student feedback, highlighting workflows that could benefit most teachers.

The Advantages of LLMs in Feedback Interpretation

Bano et al. (2023) shed light on the capabilities of LLMs, such as ChatGPT, in analyzing qualitative data, including student feedback. Their research found a significant alignment between human and LLM classifications of Alexa voice assistant app reviews, demonstrating LLMs’ ability to understand and categorize feedback effectively. This indicates that LLMs can grasp the nuances of student feedback, especially when the data is rich in specific word choices and context related to course content or teaching methodologies.

LLMs excel at processing and interpreting large volumes of text, identifying patterns, and extracting themes from qualitative feedback. Their capacity for thematic analysis at scale can assist educators in identifying common concerns, praises, or suggestions within students’ comments, tasks that might be cumbersome and time-consuming through manual efforts.

Limitations and Challenges

Despite their advantages, LLMs have limitations. Linse (2017) highlights that fully understanding the subtleties of student feedback requires more than text analysis; it demands contextual understanding and an awareness of biases. LLMs might not accurately interpret outliers and statistical anomalies, often necessitating human intervention to identify root causes.

Kastrati et al. (2021) identify several challenges in analyzing student feedback sentiment. One major challenge is accurately identifying and interpreting figurative speech, such as sarcasm and irony, which can convey sentiments opposite to their literal meanings. Additionally, many feedback analysis techniques designed for specific domains may falter when applied to the varied contexts of educational feedback. Handling complex linguistic features, such as double negatives, unknown proper names, abbreviations, and words with multiple meanings commonly found in student feedback, presents further difficulties. Lastly, there is a risk that LLMs might inadvertently reinforce biases in their training data, leading to skewed feedback interpretations.

Tools and Workflows

According to ChatGPT (OpenAI, 2024), a suggested workflow for analyzing data from course feedback forms is summarized as follows:

  1. Data Collection: Utilize tools such as Google Forms or Microsoft Forms to design and distribute course feedback forms, emphasizing open-ended questions to gather qualitative feedback from students.
  2. Data Aggregation: Employ automation to compile feedback data into a single repository, like a Google Sheet or Microsoft Excel spreadsheet, simplifying the analysis process.
  3. Initial Thematic Analysis: Import the aggregated feedback into qualitative data analysis software such as NVivo or ATLAS.ti. Use the software’s coding capabilities to identify recurring themes or sentiments in the feedback.
  4. LLM-Assisted Analysis: Engage an LLM, like OpenAI’s GPT, to further analyze the identified themes, categorize comments, and potentially uncover new themes that were not initially evident. It’s crucial to review AI-generated themes for their accuracy and relevance.
  5. Quantitative Integration: Combine qualitative insights with quantitative data from the feedback forms (e.g., ratings) using tools like Microsoft Excel or Google Sheets. This integration offers a more holistic view of student feedback.
  6. Visualization and Presentation: Apply data visualization tools such as Google Charts or Tableau to create interactive dashboards or charts that present the findings of the qualitative analysis. Employing visual aids like word clouds for common themes, sentiment analysis graphs, and charts showing thematic distribution can render the data more engaging and comprehensible.

Case Study: Minecraft Education Lesson

ChatGPT’s recommended workflow was used to analyze feedback from a recent lesson on teaching functions in Minecraft Education.

Step 1: Data Collection

A Google Forms survey was distributed to students, which comprised three quantitative five-point Likert scale questions and three qualitative open-ended questions to gather comprehensive feedback.

MCE Questionnaire

Step 2: Data Aggregation

Using Google Forms’ export to CSV feature, all survey responses were consolidated into a single file, facilitating efficient data management.

Step 3: Initial Thematic Analysis

The survey data was then imported into atlas.ti, an online thematic analysis tool with AI capabilities, to generate initial codes from the quantitative data. This process revealed several major themes, providing valuable insights from the feedback.

Results of AI Coding

Step 4: Manual Verification and Analysis

Upon reviewing the survey data manually, the main themes identified by Atlas.ti were confirmed. Additionally, this manual step highlighted specific approaches students took to solve problems presented in the lesson. Generally, the AI-generated codes were quite accurate, but a closer analysis of the comments (like the ones below) shows even more insightful student suggestions.

AI Coding

Step 5: Quantitative Integration

With both qualitative and quantitative data at hand, we bypass the need for a separate step for quantitative integration.

Step 6: LLM-Assisted Analysis and Visualization

Next, themes were further analyzed using ChatGPT’s code interpreter feature. ChatGPT helped analyze the data and summarized the aggregated data very accurately. It even provided Python code for generating additional visualizations, enhancing the interpretation of the feedback.

Python pandas code

ChatGPT’s guidance facilitated the creation of insightful visualizations such as bar charts and word clouds.

bar chart of qualitative data
Word cloud output

Python offers a wealth of data visualization libraries for even more detailed analysis (https://mode.com/blog/python-data-visualization-libraries).

Best Practices for Using LLMs in Feedback Analysis

Research by Bano et al. (2023) and insights from Linse (2017) highlight the potential of LLMs and automated text analysis tools in interpreting student course feedback. Adopting best practices for integrating these technologies is critical for educators and administrators to make informed decisions that enhance teaching quality and the student learning experience, contributing to a more responsive and dynamic educational environment. Below are several recommendations:

  1. Educators or trained administrators must review AI-generated themes and categorizations to ensure alignment with the intended context and uncover nuances possibly missed by the AI. This step is vital for identifying subtleties and complexities that LLMs may not detect.
  2. Utilize insights from both AI and human analyses to inform changes in teaching practices or course content. Then, assess whether subsequent feedback reflects the effects of these changes, thereby establishing an iterative loop for continuous improvement.
  3. Offer guidance on using Student course evaluations constructively. This involves understanding the context of evaluations, looking beyond average scores to grasp the distribution, and considering student feedback as one of several measures for assessing and enhancing teaching quality.
  4. This process should act as part of a holistic teaching evaluation system, which should also encompass peer evaluations, self-assessments, and reviews of teaching materials. A comprehensive approach offers a more precise and balanced assessment of teaching effectiveness.

References

Bano, M., Didar Zowghi, & Whittle, J. (2023). Exploring Qualitative Research Using LLMs. ArXiv (Cornell University). https://doi.org/10.48550/arxiv.2306.13298

Linse, A. R. (2017). Interpreting and using student ratings data: Guidance for faculty serving as administrators and on evaluation committees. Studies in Educational Evaluation, 54, 94–106. https://doi.org/10.1016/j.stueduc.2016.12.004

Kastrati, Z., Dalipi, F., Imran, A. S., Pireva Nuci, K., & Wani, M. A. (2021). Sentiment Analysis of Students’ Feedback with NLP and Deep Learning: A Systematic Mapping Study. Applied Sciences, 11(9), 3986. https://doi.org/10.3390/app11093986

OpenAI. (2024). ChatGPT (Feb 10, 2024) [Large language model]. https://chat.openai.com/chat

Posted on

Effective Technology Tools for K-12 CS Teachers

Technology plays a crucial role in teaching computer science and programming concepts to K-12 teachers. The most effective technology tools include interactive coding platforms such as Scratch, Snap! and Blockly. These tools provide a user-friendly interface and visual coding blocks, allowing students to learn programming concepts through hands-on activities and projects (Kashif Amanullah & Bell, 2020). Additionally, online learning platforms such as Code.org offer computer science platforms specifically designed for K-12 teachers. This blog examines various technologies used to teach CS in K-12 schools, drawing insights from a comprehensive study on visual programming languages (VPLs) and their suitability across different school levels.

Role of VPLs in K-12 Education:

VPLs like Scratch and ALICE have revolutionized CS education in schools. Scratch, developed by MIT, is particularly effective in elementary education due to its simplicity and interactive environment, making it an ideal tool for introducing programming concepts (Sáez-López et al., 2016). Although not web-based, ALICE has positively impacted all educational levels – elementary, high school, and undergraduate. Its ability to facilitate learning and enhance student confidence makes it an asset in the CS curriculum (Graczynska, 2010). In a 2019 study, do Nascimento et al. concluded that different visual programming languages (VPL) suit different school levels. The study focused on three VPLs: ALICE, Scratch, and iVProg. The findings indicate that Scratch is strongly suitable for elementary education, while ALICE is more appropriate for high school students. iVProg, on the other hand, has indications of suitability for high school and undergraduate levels.  

Enhancing Computational Thinking with Scratch

Studies have shown that Scratch’s block-based programming approach can significantly improve students’ computational thinking skills. Its integration into various disciplines through programming games and projects encourages creative problem-solving and logical reasoning among students (Stewart & Baek, 2023). In a significant study, Scratch was also found to integrate well into other subjects in the curriculum, such as math, science, and even art and history, where students achieved comprehension and application levels in Bloom’s taxonomy (Sáez-López et al., 2016).

Scratch Interface

The advantages of using Scratch in the classroom are that its intuitive drag-and-drop interface simplifies the programming process, allowing students to focus on the logic behind their creations rather than the code syntax. Overall, the visual programming approach via Scratch was effective for developing computational thinking, improving programming skills, enabling the creation of interactive projects, and supporting active learning pedagogies (Sáez-López et al., 2016). This is significant since Sun, Hu, and Zhou (2022) found that although girls in K-12 had higher computational thinking skills, they had more negative programming attitudes, which may impact their continued development in computational thinking. Visual programming may be a good strategic approach to engage females in computer science.

ALICE for STEAM Education

ALICE (which stands for Alice Learning in a Cyberworld Environment) is a free 3D programming platform developed at Carnegie Mellon University. The visual aspect of ALICE makes programming concepts more engaging and hands-on for students. Actions like loops, methods, and events correspond to actual animated motions they can see on screen. This helps concretize abstract coding notions that beginners often struggle to grasp.

ALICE Lists

Graczyńska (2010) highlights several example uses of ALICE targeted at middle school students:

  • Creating videos set to music, with lyrics displayed as subtitles. This combines coding with music appreciation and language arts.
  • Recording narration for animations, like reciting poetry in English or other languages. This boosts public speaking and foreign language skills.
  • Building simple games with sound effects and animations like fire. This makes programming exciting and fun.

After testing ALICE with students, Graczyńska found increased engagement and interest in programming and academics overall. The visual nature of ALICE also helps attract female students to computer science, where they are traditionally underrepresented.

The use of 3D visual programming tools like Alice has shown positive effects on students’ performance and attitude towards computer programming. Al-Tahat (2019) found that teaching visual programming greatly improved understanding of related concepts in object-oriented programming, making it a perfect fit for the intermediate grades.

Challenges and Future Directions:

The adoption of these technologies in K-12 computer science (CS) education has shown promise, yet challenges remain to be addressed. There is substantial evidence that incorporating VPLs into the K-12 curriculum significantly boosts female engagement (Sun et al., 2022; Graczyńska, 2010). Therefore, it is important to focus on course design that appeals to diverse learners, including females and underrepresented minorities. Additionally, ongoing research and development are necessary to keep up to date with technological progress and the changing needs of education (McGill et al., 2023). Sáez-López et al. (2016) have suggested that VPLs should be implemented across various subjects, particularly in social sciences and the arts, where their visual nature can inspire creative projects. Lastly, the successful integration of new programming tools hinges on teacher training and professional development. Teachers need robust support to acquire and apply these technologies effectively.

References

Kashif Amanullah, & Bell, T. (2020). Teaching Resources for Young Programmers: the use of Patterns. https://doi.org/10.1109/fie44824.2020.9273985

Sáez-López, J.-M., Román-González, M., & Vázquez-Cano, E. (2016). Visual programming languages integrated across the curriculum in elementary school: A two year case study using “Scratch” in five schools. Computers & Education, 97, 129–141. https://doi.org/10.1016/j.compedu.2016.03.003

Graczyńska, E. (2010). ALICE as a tool for programming at schools. Natural Science, 02(02), 124–129. https://doi.org/10.4236/ns.2010.22021

do Nascimento, M. D., Felix, I. M., Ferreira, B. M., de Souza, L. M., Dantas, D. L., de Oliveira Brandao, L., & de Oliveira Brandao, A. (2019). Which visual programming language best suits each school level? A look at Alice, iVProg, and Scratch. 2019 IEEE World Conference on Engineering Education (EDUNINE). https://doi.org/10.1109/edunine.2019.8875788

Stewart, W., & Baek, K. (2023). Analyzing computational thinking studies in Scratch programming: A review of elementary education literature. International Journal of Computer Science Education in Schools, 6(1), 35–58. https://doi.org/10.21585/ijcses.v6i1.156

Sun, L., Hu, L., & Zhou, D. (2022). Programming attitudes predict computational thinking: Analysis of differences in gender and programming experience. Computers & Education, 181, 104457. https://doi.org/10.1016/j.compedu.2022.104457

Graczyńska, E. (2010). ALICE as a tool for programming at schools. Natural Science, 02(02), 124–129. https://doi.org/10.4236/ns.2010.22021

Al-Tahat, K. (2019). The Impact of a 3D Visual Programming Tool on Students’ Performance and Attitude in Computer Programming. Journal of Cases on Information Technology, 21(1), 52–64. https://doi.org/10.4018/jcit.2019010104

Posted on

The Pros and Cons of Autograders in Programming Courses

Programming courses typically require assignments where students write code to fulfill specific specifications. In such courses, an autograder serves as an automated tool designed to assess student code submissions by conducting input and output tests. Autograders have been in existence since the inception of computer science as a field of study (Hollingsworth, 1960). More recently, with the increase of massive online programming courses hosting up to 500 students, autograders have gained popularity as an efficient means for grading programming assignments (Keuning et al., 2018). They are instrumental in student engagement (Iosup & Epema, 2014) and pivotal in providing students with constructive feedback (Keuning et al., 2018). However, like any educational technology, autograders come with their own set of advantages and disadvantages that warrant consideration. This post aims to explore the significant pros and cons of employing autograders for assessments in programming courses.

Several renowned proprietary programming autograders are currently available, including CodePost, CodeGrade, Codio, and Mimir. Each tool offers a wealth of academic programming resources, including built-in problems, user-friendly interfaces, flexible question setting, and code review capabilities. However, these companies impose a substantial annual fee on institutions, ranging from $20,000 to $100,000 CAD, for a standard school comprising 1000 students. Additionally, each student is required to pay a monthly fee between $10 and $50 CAD.

In my view, such pricing is excessive (and greedy) and contradicts the principles outlined in the computer science code of ethics, particularly when the software is intended to advance software development. As a result, many post-secondary institutions opt to develop and maintain autograders in-house, tailoring them to their specific preferences. This approach allows faculty to propose new features and enhancements, and students can also contribute suggestions for improvement.

Advantages of Autograders

One of the most compelling incentives for using an autograder is the significant time savings it offers instructors compared to manual grading. Studies indicate that autograders can assess assignments at least three to four times faster than human graders (Ihantola et al., 2010; Keuning et al., 2018). This substantial reduction in grading workload allows instructors to allocate more time to essential teaching tasks such as lesson planning, curriculum development, and providing student support and feedback. The time savings can be particularly substantial in large classes.

Autograders also benefit students by providing quicker feedback on their work. This is especially valuable in introductory programming classes, where receiving prompt results on smaller assignments can significantly enhance student learning and motivation (Keuning et al., 2018). Unlike human grading, which can take days or weeks, autograders can assess submissions within seconds or minutes and instantly inform students whether their code has passed or failed the test cases. This expedited feedback allows students to validate and refine their work much more rapidly than traditional grading methods permit.

A prevalent concern with human graders is the inconsistency in grading from one assignment to another, from one student to another, or even within a single assignment. Factors such as fatigue, emotional states, and biases can impact the quality of human grading, potentially leading to unfairness or errors. Autograders, by contrast, eliminate this subjectivity by applying uniform standards and tests to all submissions, ensuring consistent and equitable grading across the entire class, and thereby enhancing student satisfaction (Hagerer, 2021).

In courses that employ autograders, students quickly learn the necessity of writing code that meets all the autograder test cases to secure maximum assignment credit. While the efficacy of test-driven development (TDD) as a software testing methodology is debatable, this workflow provides students with experience in the TDD framework. Here, students continually run tests on their code to rectify errors and attain the desired functionality (Wang et al., 2011). Essentially, autograders compel students to consider testing as an integral part of coding, rather than merely striving to meet the minimal functional requirements.

Disadvantages of Autograders

A significant drawback of autograders, frequently cited in literature, is their inflexibility compared to human graders (Ihantola et al., 2010; Keuning et al., 2018; Wang et al., 2018). Autograders strictly apply identical test cases to all submissions without exception. Consequently, creative solutions that meet the assignment requirements but deviate from the expected implementation or output format are marked incorrect. Even a minor discrepancy such as a missing whitespace can be the difference between a pass and a fail. Unlike autograders, human graders can exercise judgment to accommodate alternative approaches.

Most autograders assess the functional correctness of student codes, evaluating output for given tests. However, programming courses also aim to instill good coding practices, such as readability, modularization, adherence to naming conventions, coherent design, and appropriate commenting, in students. Autograders do not adequately assess these crucial design and style aspects, leading students to neglect good design principles as long as their code passes the functionality tests.

Another concern is that while autograders are designed to offer students a structured means to advance their knowledge across multiple courses, achieving uniformity in their application across various courses is challenging, especially in larger institutions. Typically, post-secondary institutions employ autograders to maintain consistency across different courses, enabling students to track their progress effectively. However, in institutions where numerous faculty members teach diverse courses with varying requirements, achieving universal acceptance and use of autograders is complex. Faculty members may prefer different tools they are more comfortable with, and some might choose not to use autograders. This results in a lack of uniformity in tool usage from one course to another, creating a disjointed student experience.

Relying exclusively on autograders poses the risk of students learning to pass test cases without acquiring a deeper understanding of programming concepts and problem-solving skills. The emphasis on meeting the autograder’s criteria can lead students to adopt a procedural approach, focusing on achieving the correct output rather than understanding the underlying logic. Some might resort to a trial-and-error method, tweaking their program until it gains autograder approval. While this approach may secure the desired grades, it does not foster genuine understanding or long-term retention of knowledge. Baniassad et al. (2021) introduced a submission penalty at the University of British Columbia to discourage over-reliance on their in-house autograding tool. This adaptation exemplifies the flexibility of modifying tool requirements, a possibility uniquely available when the tool is developed in-house.

Finally, like any web-based software system, autograders can experience technical issues that lead to grading failures and student frustration. The UC Berkeley incident highlights the “single point of failure” risk where an autograder disruption blocks all grading capabilities. Unlike distributed human graders, a centralized automated grader represents a vulnerability to technical problems. Some may fail to meet deadlines through no fault of their own. Furthermore, if instructors refuse to make accommodations for autograder malfunctions, students can feel cheated and that the grading is unfairly disconnected from actual instruction. This speaks to larger concerns around over-reliance on algorithmic systems in education. Automated aids like autograders should not be seen as the sole means of assessment.

Conclusion

The existing body of research on autograders underscores that they are not a panacea for replacing human graders entirely. Instead, to optimize their advantages and mitigate their limitations, autograders are most effective when thoughtfully integrated into a course assessment strategy, complemented by manual grading where it is most beneficial. Below are some best practices for incorporating autograders effectively:

  • Employ autograders for basic functionality testing, while manually reviewing selected assignments for flexibility, creativity, and design.
  • Utilize autograders to assess the correctness of core logic, and rely on human graders to evaluate structure, style, and readability.
  • Complement autograder evaluations with human feedback on prevalent mistakes and areas requiring enhancement.
  • Impose penalties for excessive submissions to discourage over-reliance on the autograder.

Proper integration of autograders aligns with technology integration frameworks like SAMR, enhancing existing processes without entirely transforming the grading in programming courses. It also redefines the manner in which students engage with programming, introducing a more gamified approach. Like any educational technology, the value of autograders is derived from their strategic utilization within well-defined goals and contexts.

References

Hollingsworth, J. (1960). Automatic graders for programming classes. Communications of the ACM, 3(10), 528–529. https://doi.org/10.1145/367415.367422

Keuning, H., Jeuring, J., & Heeren, B. (2016). Towards a Systematic Review of Automated Feedback Generation for Programming Exercises. Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education. https://doi.org/10.1145/2899415.2899422

Iosup, A., & Epema, D. (2014). An experience report on using gamification in technical higher education. Proceedings of the 45th ACM Technical Symposium on Computer Science Education – SIGCSE ’14. https://doi.org/10.1145/2538862.2538899

Ihantola, P., Ahoniemi, T., Karavirta, V., & Seppälä, O. (2010). Review of recent systems for automatic assessment of programming assignments. Proceedings of the 10th Koli Calling International Conference on Computing Education Research – Koli Calling ’10. https://doi.org/10.1145/1930464.1930480

Hagerer, G. (2021). An Analysis of Programming Course Evaluations Before and After the Introduction of an Autograder. (n.d.). Ieeexplore.ieee.org.

 Wang, T., Su, X., Ma, P., Wang, Y., & Wang, K. (2011). Ability-training-oriented automated assessment in introductory programming course. Computers & Education, 56(1), 220–226. https://doi.org/10.1016/j.compedu.2010.08.003

Baniassad, E., Zamprogno, L., Hall, B., & Holmes, R. (2021). STOP THE (AUTOGRADER) INSANITY: Regression Penalties to Deter Autograder Overreliance. Proceedings of the 52nd ACM Technical Symposium on Computer Science Education. https://doi.org/10.1145/3408877.3432430

Posted on

Computer Science Curriculum Integration for K-9 Teachers

Introduction

With a rapidly advancing world and integrating more technology into life, it has increasingly become evident to parents, teachers, and students alike that technological literacy is essential to primary education. Parents are pushing for increased computer science instruction in elementary schools as they realize how vital this knowledge will be in preparing their children for success beyond grade school.  According to code.org (Computer Science Education Stats, n.d.), 90% of parents want their children to study computer science, but only 53% of high schools offer it in their curriculum. Research has also shown that students who take computer science courses in high school are 17% more likely to pursue higher education, with even higher percentages observed among traditionally marginalized populations such as females and Black and Latino students (Brown & Brown, n.d.).

While computer science courses provide fundamental skills like coding and web design, their significance extends far beyond that. Understanding technology equips students with data analysis capabilities and problem-solving skills applicable across various fields, extending beyond digital work environments. Consequently, many believe it is crucial to prioritize teaching computer science concepts. This post outlines my efforts to understand the computer science curriculum’s current needs and promote computer science education among elementary school teachers in British Columbia.

Assessing Current Needs

According to Dr. Shannon Thissen, Regional Administrator for Educational Technology and Computer Science Teaching and Learning in Capital Region ESD 113 in Washington, integrating computer science concepts into the curriculum poses significant challenges. Dr. Thissen mentions that many untrained teachers express concerns and seek guidance. The primary obstacle is teachers’ fear of the unknown, and overcoming this fear is crucial. Several programs, including those offered by code.org, provide free resources for teachers. However, the participation of schools in these programs in British Columbia remains limited. Dr. Thissen also highlights that teachers may hesitate to incorporate such programs into their curriculum if they are not mandated.

In a previous post, I discussed the inconsistency of computer science programs in grade schools across British Columbia, particularly emphasizing the needs of interior and rural areas. While I faced difficulties connecting with schools that desired my services, I recently connected with a Programming 11/12 teacher in Kamloops. I had the opportunity to speak to her class, providing valuable insights into the needs of both teachers and students. Students showed great interest in game development and 3D animation. Despite being a beginner-level class, students had diverse backgrounds in the subject, with some having significant experience with Unity while others struggled with the early stages of block programming. However, their shared enthusiasm for the subject validated studies showing that a majority of students enjoy computer science (Computer Science Education Stats, n.d.). This highlights the need for increased promotion of computer science in the earlier grades to nurture this enjoyment.

Currently, I am collaborating with John Knox Christian School, which is revamping its computer science and technology curriculum. Working with the Director of Curriculum and Computer Science, we are developing a series of workshops to assist K-9 teachers in integrating computer science into their classrooms. This collaboration is an excellent opportunity as John Knox controls its curriculum from K-12, enabling longer-term assessment of student success through the workshops with the same group of students.

Project Design

The central portion of the project is divided into four phases: Assessing Needs, Planning and Foundations, Integration, and Reflection.

Assessing Needs:

To ensure alignment with the school’s desired outcomes and address the specific needs of K-9 teachers, a survey will be created and distributed to all K-9 teachers. The survey aims to gather information on teachers’ objectives, knowledge levels, and areas of interest in computer science. The survey includes choices for dedicated topics such as programming, computational thinking, artificial intelligence, game development, and computer hardware. The project will tailor its objectives and content to meet the needs and interests of the teachers. The survey will be sent out during the first week of summer break.

Planning and Foundations:

This phase involves conducting a workshop covering computer science fundamentals, including vocabulary, problem-solving, and demystifying computer science. The workshop will also focus on integrating hands-on activities for teachers to experience and practice computer science concepts firsthand. Teachers will be guided in designing activities that promote problem-solving, collaboration, and critical thinking skills. Additionally, resources such as coding platforms, educational apps, and lesson plans will be provided to support teachers in implementing computer science activities beyond the workshop. By the end of the workshop, teachers should have a plan to incorporate computer science concepts into an existing lesson. This workshop is scheduled for the first professional development day in September 2023.

Integration:

Teachers will execute their plans to integrate computer science concepts into the classroom during the integration stage. One strategy for this stage is to have coaches co-teach a computer science lesson alongside the classroom teacher. Initially, the director and I will act as coaches, but the plan is to train coaches for future iterations. Co-teaching allows the teacher to observe and learn from the coach’s expertise and experience, helping teachers gain confidence and deepen their understanding. The coach can help guide students through hands-on activities and will provide positive reinforcement, celebrate teachers’ achievements, and acknowledge their efforts in integrating computer science concepts. In addition to co-teaching, coaches will observe regularly and provide teachers with constructive feedback. Classroom observations, either in person or through video recordings, will be conducted to assess the implementation of computer science lessons and the effectiveness of teaching strategies. Coaches will then provide feedback, highlighting strengths and offering suggestions for improvement. This feedback loop will enable teachers to reflect on their practice, refine their instructional techniques, and make continuous progress in integrating computer science effectively.

Reflection:

Integration is ongoing, and teachers require continued support beyond the initial stages. Follow-up meetings, workshops, or online forums will be organized to facilitate knowledge sharing, questions, and further guidance. These support mechanisms will help sustain the momentum and provide teachers with ongoing professional development and collaboration opportunities. Coaches will curate and share relevant resources, including lesson plans, coding activities, and best practices, to further support teachers’ integration efforts. This stage is scheduled for the third professional development day in November 2023.

Analyzing the Design

Using Adria Steinberg’s (1998) six A’s of evaluating project design, my proposed project aligns with these principles, ensuring the successful integration of CS into K-9 classrooms.

Academic Rigor: The project emphasizes challenging and intellectually stimulating CS content. It provides opportunities for students to engage in practical problem-solving, critical thinking, and analytical reasoning. The project also encourages teachers to adopt a different mindset when teaching CS.

Authenticity: The project provides real-world contexts and experiences that connect CS concepts to students’ lives and interests. It includes relating CS to authentic scenarios, which enhances student motivation and engagement in other subjects. By establishing these connections, the project makes CS more relevant and meaningful to students.

Applied Learning: The project emphasizes the hands-on application of CS knowledge and skills. It encourages teachers to apply CS-related skills to different activities in the classroom. This approach allows students to experiment, problem-solve, and apply CS principles in real-world contexts.

Active Exploration: The project encourages students to explore CS concepts through inquiry-based and self-reflective learning. It creates an environment that nurtures teachers’ and students’ curiosity, exploration, and independent thinking; this fosters a sense of ownership in their CS learning journeys.

Adult Relationships: The project recognizes the importance of fostering supportive and meaningful connections between students and adult mentors or educators. By incorporating coaches or mentors into the framework, the project provides students (and teachers) opportunities to interact with CS professionals, experts, or mentors. These interactions offer guidance, share real-world experiences, and serve as role models for students.

Assessment: The project includes an assessment plan to evaluate student learning and program effectiveness. It incorporates post-integration surveys and other assessment strategies to measure the impact of CS integration on student learning outcomes. Gathering feedback from stakeholders informs program improvements and ensures ongoing effectiveness.

Next Steps:

To effectively integrate computer science into K-9 classrooms, conducting one or two workshop iterations is needed. This allows for refinement and improvement of the workshop content and delivery based on feedback and insights from the initial sessions. During each iteration, it is important to collect feedback from participating teachers. Assessing the teachers’ comfort levels with integrating computer science concepts and their perceived efficacy in implementing the learned strategies in their classrooms will help gauge the workshop’s effectiveness. Additionally, student learning outcomes can be evaluated through pre- and post-workshop assessments to measure students’ knowledge growth, skills development, and attitude toward computer science. Comparing these assessments will provide evidence of the workshop’s impact on student learning.

Once the initial iterations of the workshop have been conducted and evaluated, the next step is to engage rural area schools with a more concrete plan. Building on the lessons learned and feedback from the initial workshops, it is essential to develop a targeted approach specifically tailored to the needs and challenges of rural schools. Further changes to the workshop may include solutions to overcome unique barriers such as limited resources, infrastructure challenges, or teacher training opportunities. Designing the workshop to accommodate these conditions will increase its relevance and effectiveness in rural settings.

If the workshop successfully enhances teacher comfort levels and promotes student learning, developing a Professional Development Program (PDP) at the university is an excellent opportunity. A PDP would offer a more structured and comprehensive training program for teachers seeking to integrate computer science into their K-9 classrooms. The Education Department can collaborate with the School of Computing Science and instructional design experts to design a robust PDP curriculum. This curriculum should address the specific needs of teachers, providing them with theoretical knowledge, practical skills, and ongoing support to integrate computer science concepts into their classrooms effectively.

References

Computer Science Education Stats. (n.d.). Code.org. https://code.org/promote/stats

Brown, E., & Brown, R. (n.d.). The Effect of Advanced Placement Computer Science Course Taking on College Enrollment. http://www.westcoastanalytics.com/uploads/6/9/6/7/69675515/longitudinal_study_-_combined_report_final_3_10_20__jgq_.pdf

CS Journeys Resources: Mentorship and community. (n.d.). Code.org. Retrieved June 3, 2023, from https://code.org/beyond/mentors-and-community

Steinberg, A. (1998). Real learning, real work : school-to-work as high school reform. Routledge.

K–12 Computer Science Framework. (n.d.). K12cs.org. https://k12cs.org/

Posted on

Competitive Programming Tools in the Classroom

Introduction

For young and upcoming computer scientists, competitive programming can be a powerful tool to hone essential skills. It helps sharpen problem-solving and analytical thinking abilities and provides the creative opportunity to experiment with algorithms in a safe and structured environment. With that said, introducing competitive programming into the classroom curriculum can open exciting opportunities for students of all ages, from elementary school through high school and beyond. In this blog post, we’ll take a closer look at what competitive programming is, why educators should consider bringing it into their classrooms and how they can do so successfully.

Competitive Programming and its Benefits for Students

One critical benefit of competitive programming is the development of problem-solving skills. Competitive programming challenges students to solve complex algorithmic and logical problems under pressure. This process helps enhance critical thinking and analytical skills and encourages students to approach problems from multiple angles. These skills are essential not only for programming but also for handling challenging situations. Students participating in competitive programming are exposed to different programming languages, tools, and mathematical methods, which they apply to discover new concepts and techniques. This exposure allows students to identify their strengths and interests in software development and tailor their learning to focus on these areas.

The interactive nature of competitive programming creates an ideal platform for students to develop teamwork and collaboration skills. In a team contest, students can organize themselves into teams during competitions and work together to solve problems. This process fosters a culture of collaboration, mutual respect and helps to build teamwork. Students can learn from each other to improve their coding skills and tackle complex problems requiring the cooperation of different skill sets. The competitions are rigorous and challenging, but successfully solving a difficult problem can increase a student’s confidence, self-esteem, sense of accomplishment, and motivation to participate in more challenges (Macgowan, 2015). This self-confidence can extend beyond the competition to other areas of their lives, whether in the classroom, workplace, or personal lives.

We are, of course, leaving out the most obvious – competitive programming can enhance a student’s career in the tech industry. Competitions can exhibit a student’s talent and abilities to a network of potential recruiters and employers such as Google, Microsoft, Facebook, and Apple, to name a few. Participating in competitions can increase networking opportunities, learn about job positions and companies, and prepare for recruitment. Tech giants such as AWS, IBM, and Huawei frequently sponsor international programming competitions such as ACM’s International Collegiate Programming Contest. The skills learned through competitive programming, including problem-solving, teamwork, and collaboration, are highly valued in today’s workplace and in-demand careers such as software development, data analysis, and project management.

Integrating Competitive Programming in the Classroom

Competitive programming can be a powerful learning tool for students, but finding the right resources can be overwhelming. To ensure that your students get the most out of their competitive programming lessons, it’s essential to choose resources that are challenging yet accessible, engaging, and proven to deliver results.

There are several useful resources to consider, such as textbooks, coding challenges, online forums, and programming contests. Seeking advice from experienced professionals and replicating past contests can also be helpful. When selecting resources, it’s important to consider the age appropriateness of the material and adjust the difficulty level to match the students’ skills.

Younger students can benefit from beginner-based coding platforms such as Snap (https://snap.berkeley.edu/) , CodeCombat (https://codecombat.com/) , and Tynker, as well as game-based projects from the Code Olympiad (https://www.codeolympiad.id/). These tools contain less competition and is geared more towards learning.

For middle or high school students, resources like The USACO Guide (https://usaco.guide/general/intro-cp?lang=cpp) and alGIRLithm (https://algirlithm.org/) are gentle introductions to competitive programming.

For even more advanced students, tools like vjudge (https://vjudge.net/) can be used to curate online judges and create custom contests for practice assessments, icebreaker games, or class exercises. With these resources, teachers can engage student participation, foster collaboration, and add an exciting twist to classroom activities. Watch the following video for a simple workflow on how to create a classroom contest:

Textbooks, coding challenges, online forums, and programming contests are some useful resources to consider. Seek advice from professionals in the field who use technical interviews to find the right resources for your classroom. Replicating past contests from experienced colleagues is also useful. To identify resources for competitive programming in the classroom, it is important to look for age-appropriate resources. For example, middle or high school students may benefit from resources like The USACO guide and alGIRLithm, which are gentle introductions to competitive programming. Additionally, it is important to consider the material’s difficulty level

Conclusions and Recommendations:

As we discussed in an earlier post, gamified activities, when properly used in the classroom, create an engaging and enjoyable learning experience by adding elements such as scoring, rewards, and checkpoints. Adding these features within competitive programming can help students enjoy the process of learning new algorithms, data structures, and problem-solving techniques, making it a rewarding and enjoyable experience. There must also be an element of progress in the contest. A strong sense of progress is one of the most significant benefits of gamification. Game elements such as ranks, badges, or community recognition can be incredibly motivating. In a team contest, competitive programming can help encourage collaboration and networking through various social features, such as leaderboards and chat rooms. Discussing strategies and approaches with other coders can help students get support and feedback on their work.

It’s worth noting that, despite its benefits, competitive programming is not suitable for all students. As the competitive programming community is filled with members who prioritize winning over all else and devote excessive amounts of time to these platforms, such people struggle to balance their personal and professional lives. Furthermore, competitive programming does not reflect real-world programming, as the development workflows and responsibilities involved are very different (mehulmpt, 2020). Instead, it serves as a means to an end. If you aren’t enjoying the ride, there’s a chance you won’t enjoy the outcome, either. Thus, it is not advisable to use competitive programming as an assessment tool for assignments or exams, as this would only add stress and increase feelings of competitiveness among students.

References

Macgowan, M. J., & Wong, S. E. (2015). Improving Student Confidence in Using Group Work Standards. Research on Social Work Practice, 27(4), 434–440. https://doi.org/10.1177/1049731515587557

‌mehulmpt. (2020, June 27). Mythbusting Competitive Programming – You Don’t Need to Learn It. FreeCodeCamp.org. https://www.freecodecamp.org/news/mythbusting-competitive-programming/

Zhan, Z., He, L., Tong, Y., Liang, X., Guo, S., & Lan, X. (2022). The effectiveness of gamification in programming education: Evidence from a meta-analysis. Computers and Education: Artificial Intelligence, 3, 100096. https://doi.org/10.1016/j.caeai.2022.100096

Posted on

Computer Science Curriculum in B.C.

Introduction

As an instructor of computer science at Simon Fraser University, one of my roles involves assessing the level of computer science knowledge possessed by high school graduates and the ease of their transition into higher education. These assessments help the school to evaluate the adequacy of our university’s introductory computer science courses. Unfortunately, we have observed that a considerable number of students are struggling with our first-year programming courses.

In this article, I will explore the recommendations made by the British Columbia government regarding the computer science field in grade schools. Specifically, I will investigate how these recommendations are being implemented in schools across the Lower Mainland and evaluate whether they are effective in preparing students for university-level computer science coursework. By doing this, I hope to shed light on the current state of computer science education in our region and make recommendations for improving the preparation of students for university-level computer science coursework.

Computer Science Curriculum Recommendations in K-12 Schools

There are two ways to incorporate CS concepts into a grade’s curriculum: as an entire course or as an integration of existing materials. A common misconception about computer science is that it has a bi-conditional relationship with coding, that they are one and the same. In fact, a well-designed curriculum must also include critical thinking, problem-solving, teamwork, communication skills, technical writing skills, and testing methodologies, among other vital skills. Successful implementation of a computer science curriculum not only contains coding but also equips students with other diverse tools for their future careers.

The BC government website (https://curriculum.gov.bc.ca/curriculum/adst) recommends that students from kindergarten to grade 3 are introduced to computer science basics, such as algorithms, sequencing, and problem-solving concepts, through interactive, “non-computer” activities. In grades 4-5, students move on to learn about block-based programming, granting them an entry into coding and the ability to create interactive digital media. In grades 6-7, students now apply their computational thinking skills to solve real-world problems using charts, lists, diagrams, and arrays with an introduction to computer architecture and hardware, responsible computer use, and visual programming. Finally, in grades 8-9, students learn about basic software instructions with algorithms that others can repeat, debugging algorithms, elementary modularization, binary data representation and programming languages, including visual programming.

In Grade 10, students will delve into topics such as security risks, debugging, networking, social implications, digital literacy and citizenship, and planning and writing simple programs (including games). In a separate course, it is recommended that students explore Computer Applications that center on understanding the importance of user experience. computer hardware, peripherals, internal and external components, standards, intermediate features of business applications, including word processing, spreadsheets, and presentations, operating system shortcuts and command line operations.

In addition, the B.C. government recommends a Web Development 10-course covering design opportunities, HTML and CSS, domain and hosting, copyright laws and Creative Commons usage protocols, ethics of cultural appropriation, security and privacy, and database management. While some of these areas may appear outdated, they still offer a solid understanding of web standards and communications.

In grades 11-12, students can enroll in Computer Programming 11 and 12, where they will learn various programming skills. These skills include the design cycle, error handling, debugging, problem decomposition, reading and altering code, pair programming, programming constructs such as input/output, conditions, and loops, algorithm design, functions, classes, pre-built libraries and their documentation, inline commenting to document source code, use of test cases to detect logical or semantic errors and software ethics.

In general, the suggested curriculum appears to be quite ambitious, and I have concerns about the extent and practicality of the material taught in the classroom. Several of the topics covered are typically introduced in second-level programming courses at the university level. If the high school curriculum can provide a sufficient depth of understanding, it would establish a strong foundation for many students, enabling them to tackle more advanced computer science courses without difficulty.

CS Curriculum implementations

Code.org (https://studio.code.org/courses?view=teacher) is the leading resource for computer science education, offering an excellent and well-designed curriculum to introduce students to computer science at all grade levels.

For elementary school students (grades K-5), Code.org provides CS Fundamentals. This program includes “unplugged” non-computer activities to teach computational thinking, problem solving, programming concepts, and digital citizenship.

The middle school (grades 6-8) curriculum, known as Computer Science Discoveries, builds upon the elementary school program by introducing students to more advanced concepts at an intermediate level. These include web development, communication, and problem-solving.

For high school students (grades 9-12), Code.org offers more specialized courses in computer science for students who wish to dive deeper into the subject. These include physical computing, big data, privacy, and algorithms, and advanced placement (AP) courses in Java for more advanced students.

Code.org also offers professional development courses for educators to help them effectively teach computer science. The curriculum and courses provided by Code.org are designed to help students develop computational thinking and coding skills while broadening their understanding of computer science.

Out of the 37 public high schools in Burnaby, Surrey, and New Westminster, only 4 schools use code.org as a guide, and these schools are all associated with the Advanced Placement (AP) programs. These numbers suggest that schools are aware of the usefulness of code.org materials but need more staff to implement the courses. There needs to be a standardized curriculum across these schools.

For example, the Burnaby High School website describes Computer Science 10-12 as “an introductory programming course for students with no experience.  Learn to create video games for your phone, tablet, computer, or the web.” This description is very vague and suggests that these courses do not come close to implementing the recommendations set out by the B.C. government. Other schools in Burnaby do not even offer Computer Science 11/12 courses. In contrast, New Westminster Secondary offers Computer Programming 11/12, which fully implements the government’s recommendations and more. These courses should be AP courses, depending on the depth of coverage. The difference in offerings between the two school districts is concerning for students entering higher education computer science studies as it may result in significant differences in programming knowledge.

I attempted to reach out to over 20 Computer Science teachers from different schools in the New Westminster and Burnaby areas, but unfortunately, I did not receive any responses from them. Unfortunately, there is a lack of motivation and interest in enhancing their teaching methods in CS courses. In a recent conversation with Shannon Thissen, the Regional Administrator of Educational Technology and Computer Science in Capital Region ESD 113, she confirmed that this is a common issue in all communities. She suggested that CS mentorship and coaching could alleviate teachers’ fears and uncertainties about teaching the subject.

Conclusion

The BC government’s and code.org’s recommendations for computer science education are ambitious but achievable. Higher education instructors and industry leaders should collaborate with high school teachers to strengthen and standardize the various tools and workflows currently taught in the CS curriculum.

In subsequent posts on this topic, I plan to explore various initiatives and reach out to more teachers throughout BC to gather more specific information on the curriculum and materials being taught in classrooms. Additionally, I aim to investigate the differences in implementing CS 10, 11, and 12 courses in schools across BC. It would also be interesting to compare the needs of schools in the lower mainland, which are primarily middle to upper-middle class, with those in interior communities. By doing so, we can determine if there is a significant discrepancy in the quality of computer science education and explore potential solutions to bridge the gap.