By Fenia Aivaloglou and Anna van der Meulen
When including group assignments in our courses we, as instructors, must take several decisions: should we assign the student teams, or should we let the students decide who their teammates will be? Should we be monitoring the projects’ source course repositories? How (and if) should we identify and act upon slacking members? Should we give all team members the same grade, or should we differentiate and, if the latter, how?
From prior research from the education domain, we know that work and grade equity concerns negatively impact students’ attitudes toward working in a group, and that the setup of group assignments greatly affects their success in terms of group performance, learning, collaboration patterns, and member accountability. We recognize, however, that programming assignments in computer science education have some unique characteristics: students enter our courses with very diverse levels of prior programming experience, which could impact team formation and work distribution. Practices like pair programming could affect collaboration and accountability. Moreover, while in other types of course projects the only data available to the instructors are the project deliverables, software projects can expose their process data through the source code repository or through tools such as the recently presented GitCanary.
In our study, we wanted to focus on what the students had to say. We interviewed 20 final-year bachelor’s and master’s students from four universities about their experiences on group programming assignments of varying group sizes and scopes, in which they have participated throughout their studies, and about their perceptions on assignment setup, grading and monitoring. This resulted in two main topics in our findings: work division strategies, and grading.
1. Work division strategies
Strategy in dividing up the work
Students describe that in their group assignments they usually use an approach of dividing up the work into smaller subtasks, which is worked on and completely individually by the groupmembers. This division of the work is combined with more joined practices: convening at the start of the project to discuss the assignment and brainstorm, checking back in during the course of the project, approaching groupmembers with questions, and checking each other’s work. As one student expresses: “What we do is first thing we just looked through it, we take like one day maybe to all read the assignment and ask questions if something is not clear. But when everybody reads the assignment and everything is clear, we, the first thing we do usually, is just try to split it into parts”. It differs how often such practices occur, ranging between reconvening every week to only being in touch in case of specific questions or problems. Code reviews as well occur either frequently or only on occasion. Some students describe being conscious of checking other member’s codes for mistakes, but not implementing major changes. Other than dividing up the work and proceeding individually, several (though not all) students indicate that within group assignments they engage in pair programming.
The most often mentioned underlying motivation to divide up the work is that this is the most efficient way to obtain a good grade. Other motivations that are brought forward include that this works in terms of time management and that it is a fair approach since it is clear what everyone has to do.
Strategy in assigning tasks
Within the process of assigning the subtasks of a group assignment to individual members, skills, experiences, and expertise of the group members are taken into account. Students choose subtasks based on what is familiar to them and what they prefer: “The sort of like the amount of progress any single person can make. Um, and that’s just sort of like, well, how did you split it up? Is this person better at this kind of problem or this kind of problem? Or maybe this person is just much more productive or just much better at programming or much smarter”. Specific types of activities that students prefer are also referred to: “I usually work with friends who love these algorithmic problems and then they take them and then I’ll take, you know, the user interface, the visual things, graphical work, mathematical work”.
Further, students also talk about what type of roles they can take within their group. Often there is one person taking up a leader role. When the students take in this role themselves this can be because they prefer to have an overview, or because they like to manage the process.
2. Setup and grading
Assignment set up
Students discussed three specific aspects related to assignment setup: the team formation policy (where a distinction was made between self- or instructor-assigned groups), the group composition and its influence to group work, and the degree of instructor involvement in their work strategy.
The interviews revealed that large differences between team members’ prior programming experience, skills, and commitment are considered problematic. The self-selection of the team composition was found to have advantages and disadvantages, such as missing the education value of new groups, as expressed by a student: “so of course I’d like to select them myself, but I think there’s definitely some advantage to the instructor assigning people, because then you need to work with different people and it’s actually a pretty important skill”. It was also identified that high-skilled students tend to stick together: “Um, and also, the ones that are really the ones that are really productive, they tend to find each other and they just stick to themselves.”
The analysis indicated three grading strategies experienced by the students: being assigned the same group grade, being assigned individual grades that are determined by the instructor, and grade distribution determined by the team, with perceptions about them varying greatly. Students also expressed perceptions on the meaning of the grade and the grading factors for programming assignments, including coding efficiency and skill, code functionality, and quality.
While having all team members receive the same grade was recognized as not always being fair, differentiating between the grades team members get was also found to be challenging for several reasons, one of which is that team members collude on what they present, as a student expressed: “I think it’s very difficult for a teacher to know how much work the people do, so they decide, we also decide in our group how we’re going to present things to the teacher. Um, so a lot of the time teacher just doesn’t know what’s also true.”
Independently of the grading strategy, transparency was identified as a requirement by several students. As one of them advised us, “be really transparent about how you decide what they’re graded for, even at all. Like what are the factors? Is the teamwork, is it the quality of your code? Like what are you looking at? Uh, and then in the end, explain their grade.”
Monitoring team contributions
Several practices for monitoring team contributions were discussed. Their advantages and disadvantages are summarized in the table below.
The source code repository was thought to be a good indication of member contributions, but challenging in finding representative metrics, as one of the students expressed: “it’s a really difficult subject, and that is the biggest flaw in the automation. You have to determine what is good and what is always good,” and the always good might not apply to most metrics, such as lines of code. The source code repository was found especially useful for identifying members that contribute too much, as well as slacking members: “you can review the amount of commits and the amount of lines, but I think if you do that, it’s only to see whether someone did nothing. So you can easily separate the bad apples.”
Students form all four universities described at least one experience when they had to write a report on the contributions of themselves and other team members. Students thought that this practice of peer reporting could be useful but is susceptible to colluding team members and is ineffective in informing on slacking members because of social consequences and lack of incentive. When the individual reports are not openly shared between the team members, it also lacks transparency, as one of the students expressed: “it’s also not nice failing someone without giving them a valid reason. Cause you can say, ‘yeah, but you didn’t do as much.’ They can ask ‘according to who’ and if you can’t answer that, then you’re taking away the reason that they failed the course.”
So what are the takeaways?
First, concerning work division strategies, it can be valuable to consider that by dividing up the assignment at the start and continuing individually, there is not necessarily a lot of actual collaboration taking place. If the purpose of an assignment is to also foster students’ collaboration skills, some of the joined practices they do join in (brainstorming, checking each other works) could be explicitly encouraged. Further, students also favor to follow their preferences and existing expertise in choosing which part to work on. This can contribute to premature specialization, and lessen students’ opportunity to experiment broadly in all parts of their studies. Instructors could consider being more directive here. Within the consideration of the aim of a specific assignment or course, they could either encourage students to try less familiar activities or they could foster joined practices in general, which will also contribute to students experiencing different activities within an assignment.
Second, advantages and disadvantages were identified for all described practices for setting up monitoring and grading group programming assignments. Now that we know them, we can consider them side by side with the reasons to implement a programming assignment as a group project in the first place. For example, is it for practical reasons of scaling to the large number of students? Then the advantages of the self-selection of the team and of assigning individual grades, while enforcing several practices for monitoring contributions, would outweigh their disadvantages. Does the assignment aim to support the students in learning through collaborating with others, or to increasing the students’ collaboration skills? Then maybe having instructor-selected teams and assigning the same group grade, together with less competitive monitoring strategies such as regular group meetings or presentations, would allow the members to focus on the project, get a group feeling and self-organize, without focusing on each member’s contributions.
Overall, we stress the value of explicitly considering the aim of a group assignment (including whether this includes the practice of collaboration skills) and communicating this to the students. Further, independently of the applied strategies, we should stress the need for sharing beforehand the grading criteria, as well as for transparency and feedback, requirements that were recurring in the interviews.