Joint hand-in of individual assessments

[malyzajko]

It would be nice if students could hand-in assessments (e.g. homeworks) jointly with someone else, but flexibly choose the partner on a per-assessment basis. The grading would then also happen jointly, with grades being reported for both/all students.

This is motivated by students working in groups at Uppsala university, but these groups changing (potentially) frequently during a course.

[martin-henz]

I think this is a major change. We would need a more detailed specs:

• How does a student find a partner? What access does a student have to a list of students?
• What protocol do we apply that would let a student "propose" to a partner, and that partner then to agree (or reject?) the partnership?
• Or would the partner-finding process be done externally, and admin users would upload the partner pairing?

The latter option might be the easiest to implement: Along with uploading an assessment, an admin user would (optionally) upload a partner table. Would such a solution suffice, or do you @malyzajko intend the partner-finding to happen within Source Academy?

[malyzajko]

In our scenario, the partner finding happens externally (in Canvas).

Right now, we simulate this joint submission by students writing both partner names at the top of the submission. My initial idea was to mimic this by the submitting student writing the partners name in a dedicated field, which would submit the code for the other student (i.e. copy the submission for the other student, but retain the information that the submissions are the same for grading).

I did not consider acceptance or rejection; we normally deal with issues between partners on an individual basis. E.g. an admin can unsubmit a homework for a particular student, if there is disagreement. This (in our case) will likely be less work than uploading/modifying groups for each assessment.

[martin-henz]

I see: students specify their partners in the assessment workspace. I suppose there would be a need to check whether what the students submit is internally consistent: If student A enters student B and C as team mates, then B needs to enter A and C, and C needs to enter A and B. How would possible conflicts be resolved?

How do you envision students seeing (or collaborating on) their submission? Let's say the system has the info that A, B, and C are working in a team on an assessment. Would each of the students be able to save their assessment? Or would there be a "designated" submitter? In the former case, what if two students simultaneously edit their solution and try to save?

[malyzajko]

What I had in mind is that only one student submits for the entire group (and they agree on who this is externally, similar to how group submissions work in Canvas), but each each student can work on a homework individually and save whatever progress they have. The students would agree among themselves which version gets actually submitted.

I see the issue with conflicts. At submission time the work by the non-submitting student(s) would get overwritten with my proposal - this is different from Canvas where the submissions are separate from the work-in-progress.

[RichDom2185]

I see: students specify their partners in the assessment workspace. I suppose there would be a need to check whether what the students submit is internally consistent: If student A enters student B and C as team mates, then B needs to enter A and C, and C needs to enter A and B. How would possible conflicts be resolved?

Not only that, but unless we pass some key-value pairs to limit the possible set of values submitted by the students (similar to Canvas that assigns a UUID to each student), it would be near impossible to just leave it as an open field. Example: for a partner named "John Doe", a student can write "john" or "John" or "JOHN" or "john d" or "john d." or "d., john".... if we leave it as an open field. What if there are two people that share the name "John"? What if there are two people named "John Doe"

But if we populate the field with key-value pairs (similar to some sort of autocomplete/search), then the issue you mentioned about access rights comes into play – what's the scope of this search?

My initial idea was to mimic this by the submitting student writing the partners name in a dedicated field

I don't think we should implement it this way for the issues I mentioned above. Moreover, all these access control/etc. features would make Source Academy feel more like an Learning Management System as opposed to an online environment for learning programming – we already have LMSs like Canvas that can do this better, why should we try to reinvent our own way?

Right now, we simulate this joint submission by students writing both partner names at the top of the submission.

This makes sense, and I think we should keep it this way, but add the ability for graders to mark a submission as a joint submission. Essentially, instead of leaving the system/students to do the matching, we get graders, who:

• already have the rights to view all students' names
• are humans who can easily spot all the inconsistencies/name-spelling mistakes/variations

to manually assign the submission to a list of student names (the original submitter cannot be removed from this list), and have the grades be shared amongst these people.

It would still require some significant refactoring, but I think this may be the neatest solution. We would not need an admin user to submit a table for every single grouping for every single assessment as that would be impractical for them as well.

What do you think?

[malyzajko]

I think the proposal of having graders do the matching/assignment of grades makes sense.

[CheongYeeMing]

Hi @malyzajko, the attached link contains the proposal for SA - Teams Assessment module. It will be great if you could review the proposal and let us know if there's anything to add on.

[malyzajko]

Hi @CheongYeeMing, most of the proposal looks fine to me. However, I find that only a submitting student being able to edit code to be problematic. This will effectively mean that students cannot start working on the homework individually, e.g. start exploring solutions before working on the final version together. Also group work is much less robust to one student being unavailable for a few days, e.g. due to being sick.

It seems to me that the main issue is that there is no separation between the code editor and the submitted solutions. Would it be possible to submit code that is currently in the editor, but store this version separately from the editor code? If this was the case, then any of the students in a team could edit the code independently. Once they agree on their solution, they agree which of them submits. This workflow would be very similar to submissions in Canvas, where any of the students can submit (and re-submit up until the deadline).

I think it is good that a teacher/tutor assigns the groups. It would be great if this could be done via CSV upload (in our use-case the groups will come from a Canvas environment).

[CheongYeeMing]

Hello @malyzajko,

We appreciate your valuable feedback and have carefully considered the points you raised during our discussions.

Regarding your first concern, we understand the importance of ensuring that all students have the opportunity to submit their assessments, even in unforeseen circumstances such as illness. Therefore, we have made the decision to remove the differentiation between submitting and non-submitting students. This change allows every student to participate and submit their work.

To address the distinction between the local and remote copies of code, we have devised the following solutions:

1. Revision History: We will implement a revision history feature that tracks and stores different versions of the code. Whenever a student clicks on the save button, the code in the editor will be saved in the backend. Students will have the ability to view and access previously saved versions of their code in chronological order. This ensures that accidental changes or unwanted modifications can be easily reverted by retrieving an earlier saved copy.

2. Save-Safe: Upon loading the editor, the most recent saved copy of the code will be loaded. For instance, if Student 1 saves their progress on Question 1, when Student 2 or Student 3 open the editor for the same question, they will see the copy saved by Student 1. If Student 2 decides to save their progress as well, a check will be performed to compare the loaded copy with the most recent saved copy. In this case, since Student 2 loaded the copy from Student 1 and the current copy stored in the database matches Student 1's copy, Student 2 can safely overwrite the existing copy. However, if Student 3 attempts to save their progress on Question 1, the previously loaded copy will be from Student 1, but the current copy stored in the database is Student 2's version. As a result, Student 3 will not be able to save and overwrite Student 2's work. In this scenario, Student 3 will be prompted to save their version elsewhere before reloading the page and loading Student 2's copy. This ensures data integrity and prevents unintentional overwriting of code.

Our primary focus for the upcoming MVP release this summer will be to implement the Save-Safe option, while the Revision History feature will be considered as an enhancement for future iterations of the system.

Lastly, to provide more flexibility in team assignments, we will be offering both CSV uploading and manual creation of teams. This approach allows admin/tutors to choose the method that best suits their needs and preferences.