Group programming assignments: what students say about their collaboration strategies and their experiences with grading

By Fenia Aivaloglou and Anna van der Meulen

When including group assignments in our courses we, as instructors, must take several decisions: should we assign the student teams, or should we let the students decide who their teammates will be? Should we be monitoring the projects’ source course repositories? How (and if) should we identify and act upon slacking members? Should we give all team members the same grade, or should we differentiate and, if the latter, how?

From prior research from the education domain, we know that work and grade equity concerns negatively impact students’ attitudes toward working in a group, and that the setup of group assignments greatly affects their success in terms of group performance, learning, collaboration patterns, and member accountability. We recognize, however, that programming assignments in computer science education have some unique characteristics: students enter our courses with very diverse levels of prior programming experience, which could impact team formation and work distribution. Practices like pair programming could affect collaboration and accountability. Moreover, while in other types of course projects the only data available to the instructors are the project deliverables, software projects can expose their process data through the source code repository or through tools such as the recently presented GitCanary.

In our study, we wanted to focus on what the students had to say. We interviewed 20 final-year bachelor’s and master’s students from four universities about their experiences on group programming assignments of varying group sizes and scopes, in which they have participated throughout their studies, and about their perceptions on assignment setup, grading and monitoring. This resulted in two main topics in our findings: work division strategies, and grading.

1. Work division strategies

Strategy in dividing up the work

Students describe that in their group assignments they usually use an approach of dividing up the work into smaller subtasks, which is worked on and completely individually by the groupmembers. This division of the work is combined with more joined practices: convening at the start of the project to discuss the assignment and brainstorm, checking back in during the course of the project, approaching groupmembers with questions, and checking each other’s work. As one student expresses: “What we do is first thing we just looked through it, we take like one day maybe to all read the assignment and ask questions if something is not clear. But when everybody reads the assignment and everything is clear, we, the first thing we do usually, is just try to split it into parts”. It differs how often such practices occur, ranging between reconvening every week to only being in touch in case of specific questions or problems. Code reviews as well occur either frequently or only on occasion. Some students describe being conscious of checking other member’s codes for mistakes, but not implementing major changes. Other than dividing up the work and proceeding individually, several (though not all) students indicate that within group assignments they engage in pair programming.

The most often mentioned underlying motivation to divide up the work is that this is the most efficient way to obtain a good grade. Other motivations that are brought forward include that this works in terms of time management and that it is a fair approach since it is clear what everyone has to do.

Table 1: overview of work division strategies

Strategy in assigning tasks

Within the process of assigning the subtasks of a group assignment to individual members, skills, experiences, and expertise of the group members are taken into account. Students choose subtasks based on what is familiar to them and what they prefer: “The sort of like the amount of progress any single person can make. Um, and that’s just sort of like, well, how did you split it up? Is this person better at this kind of problem or this kind of problem? Or maybe this person is just much more productive or just much better at programming or much smarter”. Specific types of activities that students prefer are also referred to: “I usually work with friends who love these algorithmic problems and then they take them and then I’ll take, you know, the user interface, the visual things, graphical work, mathematical work”.

Further, students also talk about what type of roles they can take within their group. Often there is one person taking up a leader role. When the students take in this role themselves this can be because they prefer to have an overview, or because they like to manage the process.

Table 2: task allocation strategies

2. Setup and grading

Assignment set up

Students discussed three specific aspects related to assignment setup: the team formation policy (where a distinction was made between self- or instructor-assigned groups), the group composition and its influence to group work, and the degree of instructor involvement in their work strategy. 

Table 3: student perceptions on assignment setup

The interviews revealed that large differences between team members’ prior programming experience, skills, and commitment are considered problematic. The self-selection of the team composition was found to have advantages and disadvantages, such as missing the education value of new groups, as expressed by a student: “so of course I’d like to select them myself, but I think there’s definitely some advantage to the instructor assigning people, because then you need to work with different people and it’s actually a pretty important skill”. It was also identified that high-skilled students tend to stick together: “Um, and also, the ones that are really the ones that are really productive, they tend to find each other and they just stick to themselves.”

Grading Strategies

The analysis indicated three grading strategies experienced by the students: being assigned the same group grade, being assigned individual grades that are determined by the instructor, and grade distribution determined by the team, with perceptions about them varying greatly. Students also expressed perceptions on the meaning of the grade and the grading factors for programming assignments, including coding efficiency and skill, code functionality, and quality.

Table 4: student perceptions on grading strategies

While having all team members receive the same grade was recognized as not always being fair, differentiating between the grades team members get was also found to be challenging for several reasons, one of which is that team members collude on what they present, as a student expressed: “I think it’s very difficult for a teacher to know how much work the people do, so they decide, we also decide in our group how we’re going to present things to the teacher. Um, so a lot of the time teacher just doesn’t know what’s also true.”

Independently of the grading strategy, transparency was identified as a requirement by several students. As one of them advised us, “be really transparent about how you decide what they’re graded for, even at all. Like what are the factors? Is the teamwork, is it the quality of your code? Like what are you looking at? Uh, and then in the end, explain their grade.”

Monitoring team contributions

Several practices for monitoring team contributions were discussed. Their advantages and disadvantages are summarized in the table below.

Table 5: student perceptions on project monitoring practices

The source code repository was thought to be a good indication of member contributions, but challenging in finding representative metrics, as one of the students expressed: “it’s a really difficult subject, and that is the biggest flaw in the automation. You have to determine what is good and what is always good,” and the always good might not apply to most metrics, such as lines of code. The source code repository was found especially useful for identifying members that contribute too much, as well as slacking members: “you can review the amount of commits and the amount of lines, but I think if you do that, it’s only to see whether someone did nothing. So you can easily separate the bad apples.”

Students form all four universities described at least one experience when they had to write a report on the contributions of themselves and other team members. Students thought that this practice of peer reporting could be useful but is susceptible to colluding team members and is ineffective in informing on slacking members because of social consequences and lack of incentive. When the individual reports are not openly shared between the team members, it also lacks transparency, as one of the students expressed: “it’s also not nice failing someone without giving them a valid reason. Cause you can say, ‘yeah, but you didn’t do as much.’ They can ask ‘according to who’ and if you can’t answer that, then you’re taking away the reason that they failed the course.”

So what are the takeaways?

First, concerning work division strategies, it can be valuable to consider that by dividing up the assignment at the start and continuing individually, there is not necessarily a lot of actual collaboration taking place. If the purpose of an assignment is to also foster students’ collaboration skills, some of the joined practices they do join in (brainstorming, checking each other works) could be explicitly encouraged. Further, students also favor to follow their preferences and existing expertise in choosing which part to work on. This can contribute to premature specialization, and lessen students’ opportunity to experiment broadly in all parts of their studies. Instructors could consider being more directive here. Within the consideration of the aim of a specific assignment or course, they could either encourage students to try less familiar activities or they could foster joined practices in general, which will also contribute to students experiencing different activities within an assignment.

Second, advantages and disadvantages were identified for all described practices for setting up monitoring and grading group programming assignments. Now that we know them, we can consider them side by side with the reasons to implement a programming assignment as a group project in the first place. For example, is it for practical reasons of scaling to the large number of students? Then the advantages of the self-selection of the team and of assigning individual grades, while enforcing several practices for monitoring contributions, would outweigh their disadvantages. Does the assignment aim to support the students in learning through collaborating with others, or to increasing the students’ collaboration skills? Then maybe having instructor-selected teams and assigning the same group grade, together with less competitive monitoring strategies such as regular group meetings or presentations, would allow the members to focus on the project, get a group feeling and self-organize, without focusing on each member’s contributions.

Overall, we stress the value of explicitly considering the aim of a group assignment (including whether this includes the practice of collaboration skills) and communicating this to the students. Further, independently of the applied strategies, we should stress the need for sharing beforehand the grading criteria, as well as for transparency and feedback, requirements that were recurring in the interviews.

Are you interested to know more? Read the full paper on collaboration strategies, and the full paper on assignment set up and grading.

How is programming taught in code clubs? We asked the instructors, and this is what we found

(reposted from here)

Programming is taught everywhere: at schools, independent code clubs, Code Clubs, CoderDojos, Girls Who Code sessions, after school programs, coding summer camps, workshops in libraries and museums. At schools, the lessons are given by teachers, following national curricula. However, what about the lessons at code clubs? Who teaches, what is taught, what material is being used, and how are children learning programming? What do children find difficult, and are there any differences between boys and girls?

To find answers to our questions, we designed and run a survey. We invited code club instructors to answer through Twitter, Facebook groups of code clubs, Slack channels, newsletters and even direct emails. We collected 98 responses, we analysed them, and this is what we found.

Participants, lesson material and style

Many of the code club instructors who answered our survey have a main degree related to computer science (49%) or other STEM-related field (10%). Only a small group has a background education (18%). Almost half have no education experience, while the majority of the instructors (90%) can program in at least one commercial programming language.

Most code clubs have a small number of participants of varying ages. Female students are under-represented at code clubs; the average percentage of female students is 30%.

Most of the instructors replied that their code club is part of a program, mainly CoderDojo (36%) or Code Club (31%). The students attending Code Club classes are younger and more often female than in CoderDojo clubs.

The most commonly taught language is Scratch (89%), followed by Python in almost half of the code clubs, followed by Arduino, Mindstorms, Micro:bit, HTML, Java, JavaScript, Blockly, C-like languages (e.g. ArduC, RoboC or NXT-C). Within the taught languages we also found Sonic Pi, Blender, Snap!, Swift, MBlock, Spheros, Flowol4, Crumble, Codebug, Node JS, Lightbot and A.l.e.x.

Regarding lesson material, half of the teachers responded not using a specific lesson plan. Some declared using Code.org lessons or the Scratch Creative Computing Handbook. The students mostly work independently on their own projects (71%) and, in some code clubs, the teachers give plenary sessions. Formal assessments and grading are rare; more common are stickers or badges for achievements (47%) and other forms of formative assessment.

What do the students struggle with?

Instructors most commonly identified debugging/error messages and abstract thinking as the main difficulties. A teacher whose students are from 8 to 12 years old related the age of the students to that, reporting that (translated from Dutch) “sometimes children that register for my workshops are too young and find abstract thinking too difficult to really understand what they are doing.”. In terms of programming concepts, variables and functions were identified as the most difficult for their students.

Instructors also identified struggles related to creativity, for example “thinking for themselves instead of blindly following the tutorial” and “design something for themselves and implement that.” Related to concentration, a teacher reported that students get “distracted by playing games.” Another teacher reported the student focus on language as a learning barrier, as students “often become focussed on learning Scratch itself, rather than building higher-order skills.”

How are boys different than girls?

Teachers identified gender differences especially for two traits, namely confidence and concentration. In the question “Who is more confident?”, 50% of the responses lean towards boys. A teacher added that “I get initial “I will never understand this” reactions way more from girls than from boys. Completely invalidated after an hour or so of course, but still saddens me.” This is worrying, because gender differences related to confidence could have several implications. In prior work we have found that, especially for female elementary school students, self-efficacy is strongly correlated with how attractive they view computer science as a career path.

On the other hand, in the question “Who seems to concentrate better?” 65% of the responses lean towards girls. A teacher explained that  “Girls tend to stay on-task more, whereas some boys can be easily distracted”.

Gender differences were also reported in the preferred type of projects, with girls preferring  storytelling and visual/creative exercises, and boys preferring to implements games. Girls were identified as more responsive to instruction, whereas “Boys just start blindly without reading lessons and then run into trouble pretty quickly, then call for help. Girls tend to focus more, start reading and ask questions when they’re really stuck.”

Collaboration skills are described to be increased for girls, as “Girls tend to talk and discuss more when working in partnerships whereas boys tend to have one who takes the lead”, as well as focus (“All of the girls in my club have always been more careful and methodical. They seem to want to understand what they are doing more and don’t mind taking their time.”).

Want to know more? Read our paper, or come and see us present it at Koli Calling.




“When I grow up I want to become a programmer!”, Elli (11) – How?!

(the answer is at the end of the post)

Computer Science is not the typical profession of choice for elementary school students. This might be because most children at this age do not yet understand what computer science is. This is about to change, since an increasing number of countries is currently enriching their elementary school education with computing and programming courses. But, what will make those young students perform well in programming classes? And what will their view of programming as a career path be after they have followed their first programming course?

Finding out

To answer our questions, with Felienne Hermans we run experimental programming courses, teaching Scratch to four groups of 8 to 12 year-old students of two elementary schools in the Netherlands. We gave a total of eight lessons to each group, following the lesson plan of the Scratch MOOC on edX. During the lessons, we measured factors that have been shown to affect adult, university-level students. Those factors include:

  • Self-efficacy, or the students’ beliefs in their abilities. In education research, self-efficacy is recognized as one of the most important factors related to learning performance, and has been found to affect the choice of college major. (“I will become what I am good at.”)
  • Extrinsic (external) motivation, or motivation inspired by influences outside of the individual (for example, from other people). (“I will choose what is trendy/what my peers choose.”)
  • Intrinsic (internal) motivation, or motivation that comes from within the individual as a self-desire to learn. (“I will choose it because I like it.”)
  • The stereotypes that students assume for computer scientists, and their fit within those. We examined four stereotypical traits that have been found to apply to computer scientists [17]: Singularly focused, indicating that computer science requires an obsession with it, asocial, indicating that computer scientists have limited social skills, competitive, and male. (“I will choose what my personality fits in.”)
  • The characteristics of the students, including their age, gender and programming experience prior to the course.

Results

We found that having previous programming experience was a strong factor, correlated to extrinsic motivation, self-efficacy and CS career orientation. While it is known that prior programming experience has a  strong effect on the performance of university-level students, in our study there is an additional reason for this observation: that programming experience before our experimental course could have been obtained only through home-based or extra-curricular activities. Therefore, having prior programming experience indicates that the students had sought to learn programming themselves, or were encouraged by their environment towards programming, and this proved to give them a significant head start.

Our study highlighted gender differences, with the CS career orientation of the girls being significantly correlated with their self-efficacy, an effect which was not as strong for boys.

We also found that students’ intrinsic and extrinsic motivation are important factors, strongly correlated with their self-efficacy and, for the case of intrinsic motivation, their inclination towards a CS career.

Equally interesting is what we did not observe: course performance and stereotypical beliefs for computer scientists had no significant effect on CS career orientation. Students actually appeared unaffected by the four stereotypical traits that we studied. Even the Male trait was not assumed, with students favoring neutral or their own gender as typical for the profession. This might be attributed to their age; we could assume that they are yet too young to believe anything in particular about stereotypical traits of computer scientists. It also leaves us hopeful for the future of women in CS.

The answer

To sum up, Elli(11), probably did not wait for school to teach her programming, she had started already by herself. She likes the challenges of programming and she believes that she is good at it. Also, she has not watched movies about programmers yet, and she better leave it for after she chooses college major.

Are you interested to know more? Read our paper or come see us present it at SIGCSE 2019.

***Reposted from here***