Tag: teaching

Posted on

Models to Measure Students’ Learning in Computer Science

As computer science becomes integrated into K-12 education systems worldwide, educators and researchers continuously search for effective methods to measure and understand students’ learning levels in this field. The challenge lies in developing reliable and comprehensive assessment models that accurately and discreetly gauge student learning. Teachers must assess learning to support students’ educational needs better. Similarly, students and parents expect schools to document students’ proficiency in computing and their practical application. Unlike conventional subjects such as math and science, very few relevant assessments are available for K-12 CS education. This article explores specific models used to measure knowledge in various CS contexts and then examines several examples of student learning indicators in computer science.

Randomized Controlled Trials and Measurement Techniques

An innovative approach to measuring student performance in computer science education involves evaluating the effectiveness of teaching parallel programming concepts. Research by Daleiden et al. (2020) focuses on assessing students’ understanding and application of these concepts.

The Token Accuracy Map (TAM) technique supplements traditional empirical analysis methods, such as timings, error counting, or compiler errors, which often need more depth in analyzing the cause of errors or providing detailed insights into specific problem areas encountered by students. The study applied TAM to examine student performance across two parallel programming paradigms: threads and process-oriented programming based on Communicating Sequential Processes (CSP), measuring programming accuracy through an automated process.

The TAM approach analyzes the accuracy of student-submitted code by comparing it against a reference solution using a token-based comparison. Each element of the code, or “token,” is compared to determine its correctness, and the results are aggregated to provide an overall accuracy score ranging from 0% to 100%. This scoring system reflects the percentage of correctness, allowing for a detailed examination of which students intuitively understand specific elements of different programming paradigms or are more likely to implement them correctly.

This approach extends error counts, offering insights into students’ mistakes at a granular level. Such detailed analysis enables researchers and educators to identify specific programming concepts requiring further clarification or alternative teaching approaches. Additionally, TAM can highlight the strengths and weaknesses of different programming paradigms from a learning perspective, thereby guiding curriculum development and instructional design.

Competence Structure Models in Informatics

Torsten et al. (2015) introduced a new model in their discussion aimed at developing a competence structure model for informatics with a focus on system comprehension and object-oriented modelling. This model, part of the MoKoM project (Modeling and Measurement of Competences in Computer Science Education), seeks to create a competence structure model that is both theoretically sound and empirically validated. The project’s goals include identifying essential competencies in the field, organizing them into a coherent framework, and devising assessments to measure them accurately. The study employed the Item Response Theory (IRT) evaluation methodology to construct the test instrument and analyze survey data.

The initial foundation of the competence model was based on theoretical concepts from international syllabi and curricula, such as the ACM’s “Model Curriculum for K-12 Computer Science” and expert papers on software development. This framework encompasses cognitive and non-cognitive skills pertinent to computer science, especially emphasizing system comprehension and object-oriented modelling.

The study further included conducting expert interviews using the Critical Incident Technique to validate the model’s applicability to real-world scenarios and its empirical accuracy. This method was instrumental in pinpointing and defining the critical competencies needed to apply and understand informatics systems. It also provided a detailed insight into student learning in informatics, identifying specific strengths and areas for improvement.

Limitations

The limitation of this approach is its specificity, which may hinder scalability to broader contexts or different courses. Nonetheless, the findings indicate that detailed, granular measurements can offer valuable insights into the nature and types of students’ errors and uncover learning gaps. The resources mentioned subsequently propose a more general strategy for assessing learning in computer science.

Evidence-centred Design for High School Introductory CS Courses

Another method for evaluating student learning in computer science involves using Evidence-Centered Design (ECD). Newton et al. (2021) demonstrate the application of ECD to develop assessments that align with the curriculum of introductory high school computer science courses. ECD focuses on beginning with a clear definition of the knowledge, skills, and abilities students are expected to gain from their coursework, followed by creating assessments that directly evaluate these outcomes.

The approach entails specifying the domain-specific tasks that students should be capable of performing, identifying the evidence that would indicate their proficiency, and designing assessment tasks that would generate such evidence. The model further includes an analysis of assessment items for each instructional unit, considering their difficulty, discrimination index, and item type (e.g., multiple-choice, open-ended, etc.). This analysis aids in refining the assessments to gauge student competencies and understanding more accurately.

This model offers a more precise measurement of student learning by ensuring that assessments are closely linked to curriculum objectives and learning outcomes.

Other General Student Indicators

The Exploring Computer Science website, a premier resource for research on indicators of student learning in computer science, identifies several key metrics for understanding concepts within the field:

  • Student-Reported Increase in Knowledge of CS Concepts: Students are asked to self-assess their knowledge in problem-solving techniques, design, programming, data analysis, and robotics, rating their understanding before and after instruction.
  • Persistent Motivation in Computer Problem Solving: This self-reported measure uses a 5-point Likert scale to evaluate students’ determination to tackle computer science problems. Questions include, “Once I start working on a computer science problem or assignment, I find it hard to stop,” and “When a computer science problem arises that I can’t solve immediately, I stick with it until I find a solution.”
  • Student Engagement: This metric again relies on self-reporting to gauge a student’s interest in further pursuing computer science in their studies. It assesses enthusiasm and inclination towards the subject.
  • Use of CS Vocabulary: Through pre- and post-course surveys, students respond to the prompt: “What might it mean to think like a Computer Scientist?”. Responses are analyzed for the use of computer science-related keywords such as “analyze,” “problem-solving,” and “programming.” A positive correlation was found between CS vocabulary use and self-reported CS knowledge levels.

Comparing the Models

Each model discussed provides distinct benefits but converges on a shared objective: to gauge precisely students’ understanding of computer science. The Evidence-Centered Design (ECD) model is notable for its methodical alignment assessments with educational objectives, guaranteeing that evaluations accurately reflect the intended learning outcomes. Conversely, the randomized controlled trial and innovative measurement technique present a solid approach for empirically assessing the impact of instructional strategies on student learning achievements. Finally, the competence structure model offers an exhaustive framework for identifying and evaluating specific competencies within a particular field, like informatics, ensuring a thorough understanding of student abilities. As the field continues to evolve, so will our methods for measuring student success.

References

Daleiden, P., Stefik, A., Uesbeck, P. M., & Pedersen, J. (2020). Analysis of a Randomized Controlled Trial of Student Performance in Parallel Programming using a New Measurement Technique. ACM Transactions on Computing Education, 20(3), 1–28. https://doi.org/10.1145/3401892

Magenheim, J., Schubert, S., & Schaper, N. (2015). Modelling and measurement of competencies in computer science education. KEYCIT 2014: key competencies in informatics and ICT, 7(1), 33-57.

Newton, S., Alemdar, M., Rutstein, D., Edwards, D., Helms, M., Hernandez, D., & Usselman, M. (2021). Utilizing Evidence-Centered Design to Develop Assessments: A High School Introductory Computer Science Course. Frontiers in Education, 6. https://doi.org/10.3389/feduc.2021.695376

Posted on

Competitive Programming Tools in the Classroom

Introduction

For young and upcoming computer scientists, competitive programming can be a powerful tool to hone essential skills. It helps sharpen problem-solving and analytical thinking abilities and provides the creative opportunity to experiment with algorithms in a safe and structured environment. With that said, introducing competitive programming into the classroom curriculum can open exciting opportunities for students of all ages, from elementary school through high school and beyond. In this blog post, we’ll take a closer look at what competitive programming is, why educators should consider bringing it into their classrooms and how they can do so successfully.

Competitive Programming and its Benefits for Students

One critical benefit of competitive programming is the development of problem-solving skills. Competitive programming challenges students to solve complex algorithmic and logical problems under pressure. This process helps enhance critical thinking and analytical skills and encourages students to approach problems from multiple angles. These skills are essential not only for programming but also for handling challenging situations. Students participating in competitive programming are exposed to different programming languages, tools, and mathematical methods, which they apply to discover new concepts and techniques. This exposure allows students to identify their strengths and interests in software development and tailor their learning to focus on these areas.

The interactive nature of competitive programming creates an ideal platform for students to develop teamwork and collaboration skills. In a team contest, students can organize themselves into teams during competitions and work together to solve problems. This process fosters a culture of collaboration, mutual respect and helps to build teamwork. Students can learn from each other to improve their coding skills and tackle complex problems requiring the cooperation of different skill sets. The competitions are rigorous and challenging, but successfully solving a difficult problem can increase a student’s confidence, self-esteem, sense of accomplishment, and motivation to participate in more challenges (Macgowan, 2015). This self-confidence can extend beyond the competition to other areas of their lives, whether in the classroom, workplace, or personal lives.

We are, of course, leaving out the most obvious – competitive programming can enhance a student’s career in the tech industry. Competitions can exhibit a student’s talent and abilities to a network of potential recruiters and employers such as Google, Microsoft, Facebook, and Apple, to name a few. Participating in competitions can increase networking opportunities, learn about job positions and companies, and prepare for recruitment. Tech giants such as AWS, IBM, and Huawei frequently sponsor international programming competitions such as ACM’s International Collegiate Programming Contest. The skills learned through competitive programming, including problem-solving, teamwork, and collaboration, are highly valued in today’s workplace and in-demand careers such as software development, data analysis, and project management.

Integrating Competitive Programming in the Classroom

Competitive programming can be a powerful learning tool for students, but finding the right resources can be overwhelming. To ensure that your students get the most out of their competitive programming lessons, it’s essential to choose resources that are challenging yet accessible, engaging, and proven to deliver results.

There are several useful resources to consider, such as textbooks, coding challenges, online forums, and programming contests. Seeking advice from experienced professionals and replicating past contests can also be helpful. When selecting resources, it’s important to consider the age appropriateness of the material and adjust the difficulty level to match the students’ skills.

Younger students can benefit from beginner-based coding platforms such as Snap (https://snap.berkeley.edu/) , CodeCombat (https://codecombat.com/) , and Tynker, as well as game-based projects from the Code Olympiad (https://www.codeolympiad.id/). These tools contain less competition and is geared more towards learning.

For middle or high school students, resources like The USACO Guide (https://usaco.guide/general/intro-cp?lang=cpp) and alGIRLithm (https://algirlithm.org/) are gentle introductions to competitive programming.

For even more advanced students, tools like vjudge (https://vjudge.net/) can be used to curate online judges and create custom contests for practice assessments, icebreaker games, or class exercises. With these resources, teachers can engage student participation, foster collaboration, and add an exciting twist to classroom activities. Watch the following video for a simple workflow on how to create a classroom contest:

Textbooks, coding challenges, online forums, and programming contests are some useful resources to consider. Seek advice from professionals in the field who use technical interviews to find the right resources for your classroom. Replicating past contests from experienced colleagues is also useful. To identify resources for competitive programming in the classroom, it is important to look for age-appropriate resources. For example, middle or high school students may benefit from resources like The USACO guide and alGIRLithm, which are gentle introductions to competitive programming. Additionally, it is important to consider the material’s difficulty level

Conclusions and Recommendations:

As we discussed in an earlier post, gamified activities, when properly used in the classroom, create an engaging and enjoyable learning experience by adding elements such as scoring, rewards, and checkpoints. Adding these features within competitive programming can help students enjoy the process of learning new algorithms, data structures, and problem-solving techniques, making it a rewarding and enjoyable experience. There must also be an element of progress in the contest. A strong sense of progress is one of the most significant benefits of gamification. Game elements such as ranks, badges, or community recognition can be incredibly motivating. In a team contest, competitive programming can help encourage collaboration and networking through various social features, such as leaderboards and chat rooms. Discussing strategies and approaches with other coders can help students get support and feedback on their work.

It’s worth noting that, despite its benefits, competitive programming is not suitable for all students. As the competitive programming community is filled with members who prioritize winning over all else and devote excessive amounts of time to these platforms, such people struggle to balance their personal and professional lives. Furthermore, competitive programming does not reflect real-world programming, as the development workflows and responsibilities involved are very different (mehulmpt, 2020). Instead, it serves as a means to an end. If you aren’t enjoying the ride, there’s a chance you won’t enjoy the outcome, either. Thus, it is not advisable to use competitive programming as an assessment tool for assignments or exams, as this would only add stress and increase feelings of competitiveness among students.

References

Macgowan, M. J., & Wong, S. E. (2015). Improving Student Confidence in Using Group Work Standards. Research on Social Work Practice, 27(4), 434–440. https://doi.org/10.1177/1049731515587557

‌mehulmpt. (2020, June 27). Mythbusting Competitive Programming – You Don’t Need to Learn It. FreeCodeCamp.org. https://www.freecodecamp.org/news/mythbusting-competitive-programming/

Zhan, Z., He, L., Tong, Y., Liang, X., Guo, S., & Lan, X. (2022). The effectiveness of gamification in programming education: Evidence from a meta-analysis. Computers and Education: Artificial Intelligence, 3, 100096. https://doi.org/10.1016/j.caeai.2022.100096

Posted on

How Effective are Pair Programming Tools for one-on-one Collaboration with Students and Colleagues?

Introduction

The practice of pair programming is a software development methodology that involves two programmers working jointly on one workstation to accomplish a shared objective. The two programmers involved in the pairing exercise work together to solve a single problem, with one functioning as the driver whose primary task is to write the code, while the other acts as the observer responsible for proposing potential improvements to the code and thinking strategically about the big picture.

The driver and observer’s roles are often rotated at predetermined intervals to keep the pairing exercise productive and engaging. This periodic rotation of roles is intended to prevent one of the programmers from becoming overly exhausted or unfocused during the task. The benefits of pair programming are to generate well-designed and standardized code while improving productivity, knowledge-sharing, and collaboration among team members. It also facilitates rapid problem-solving and reduces the probability of code errors. Moreover, it allows team members to learn from each other, share expertise and knowledge, and develop better communication skills.

The pandemic forced many software developers to transition to remote work, resulting in the need for technological advancements to facilitate remote software engineering workflows. One such innovation was an extension known as “Live Share,” which was developed specifically for remote pair programming. The following video shows the workflow of the “Live Share” extension.  It also highlights several main features:

Live Share Features

Applying LiveShare to Teaching

Since the process and workflows are very similar, I aim to delve into the effectiveness of utilizing this extension for connecting with students on a one-on-one basis as well as collaborating with colleagues on teaching materials. It is worth noting that these use cases are not the primary focus of the extension. Nevertheless, I am interested in exploring the potential benefits of using the “Live Share” extension in the following scenarios:

Scenario 1: Collaborating with Colleagues

A computer science coach is tasked with helping her computer science teachers create materials for their classes. In order to accomplish this, she uses the “Live Share” extension to work remotely with a teacher to develop and refine class materials in real time. Using this strategy, teachers and coaches can more effectively foresee problems and ensure students have the support they need to succeed.

One of the key benefits of using “Live Share” in this context is the ability to work together on specific problems that students may encounter during class. For example, if there is a coding assignment that many students are struggling with, the teacher and coach can collaborate in real-time to find possible solutions and prepare debugging strategies for anticipated student problems. This can be incredibly valuable, as it allows the teacher to work through issues quickly and efficiently, ensuring that students are able to keep up with the pace of the class.

Scenario 2:

A group of five students in a class are working on a programming assignment, and the teacher monitors their progress remotely while giving live feedback. The teacher uses “Live Share” and connects to her each of her student’s IDEs remotely.  During the live session, the teacher can monitor the progress of each of her students while giving feedback using the chat functionality.  She is also able to help students debug by writing hints into their code whenever they’re stuck.  Furthermore, she is able to insert breakpoints for her students remotely to help them with the debugging process.

By examining the potential advantages and limitations of utilizing the “Live Share” extension in these scenarios, we can gain a deeper understanding of its potential as a tool for enhancing remote teaching and learning in the field of computer science. The following video shows a high-level workflow of a teacher connecting remotely with a student as an observer and engaging in a short lesson:

Remote Teaching with LiveShare on Visual Studio Code

General analysis:

During my two-hour experience with “live share,” I was pleasantly surprised by the tool’s exceptional features that aides in code reviews and debugging. The tool allows multiple programmers to work simultaneously on the same codebase, which proved to be immensely helpful, particularly when dealing with intricate problems. It streamlines the process, making it more efficient and allowing for effective team collaboration. Whether you are working on a problem concurrently with a team member, seeking feedback on your code, or looking for debugging help, this tool will certainly prove to be beneficial.

One standout feature that I came across was the capacity to share a command-line terminal. This feature, among other things, enables remote users to access the local database of the driver, which is extremely useful for troubleshooting issues that may arise during pair programming or mob programming. The real-time collaboration aspect of this feature allowed multiple developers to work together on a data model simultaneously.

Limitations and Final Thoughts

It’s worth mentioning that the extension “live share” was initially created for pair programming. As a result, some of the difficulties I have experienced when using it for teaching may be due to the fact that it was not designed for this specific use case. However, some limitations are still worth discussing when using the app for teaching purposes.

The lack of an integrated voice connection feature can be a challenge. While users can rely on external tools like Zoom, having a built-in audio feature for real-time problem-solving during code reviews or debugging would be more convenient. Although many users have requested that this feature be returned on the extension’s website, there are currently no plans.

Secondly, the observer’s IDE cannot access the driver’s debugger. While this limitation makes sense from a pair programming perspective, it poses a challenge when using the app as a teaching tool since debugging is an important part of teaching new programmers an environment or language.

Finally, setting up the extension requires technical knowledge, which can be difficult for students who lack technical proficiency. It is challenging to differentiate between a local and a production server, particularly when teaching languages or frameworks that involve local servers. Therefore, the app is best suited for teaching relatively simple languages like Python to semi-technical students rather than beginners.