search

devhub-tud / devhub

DevHub is a software system designed to give students a simple practical introduction into modern software development.
15 stars 8 forks source link

Improve algorithm to assign reviewers #489

Open TimvdLippe opened 7 years ago

TimvdLippe commented 7 years ago

In #397 TA's could be assigned to groups. An algorithm was introduced to automatically distribute the TA's over all the groups. I would like to request two improvements:

  1. Add the option to specify how many groups a teacher/TA should be able to review. Right now, Mauricio (teacher) has groups assigned, which should not happen. Also Liam (who is admin) did not get any groups assigned, but he should have. Also Jurriaan was TA on DevHub, but should not have assigned any group. Lastly, Liam and me should have fewer groups than other TAs, to make sure we remain in our FlexDelft budget (we TA more lab hours).
  2. In every year, a TA should not be allowed to grade the same group twice. E.g. if a TA graded group 42 for assignment 1, he/she should not be able to grade that group for assignments 2 and 3.

These improvements are required to achieve feature parity with the sheet we use right now. Since the distribution of TAs was also pushed on DevHub, TAs were confused and graded the "wrong" groups. As such, the grading of assignment 3 has significantly be delayed due to confusion and wrong assignments (to for example Jurriaan who has a lot of groups on DevHub ungraded, but should not have been).

jwgmeligmeyling commented 7 years ago

Mauricio should not have been listed as TA, he should have been a (course) admin. Liam should have been listed as TA (which we didn't do back then due to a bug). Jurriaan actually was a TA and did grade some groups, but. was not scheduled in the TA roster and didn't like just doing the bad parts a lot. So assuming that TA's have to grade groups is just fine, but we just had so set up some things differently.

The real question here is not whether we want to be able to specify our preference in the number of groups we have to grade, but whether its actually preferable to do so. Because all TA's had exactly the same budget and are expected to provide an equal contribution. In all previous editions, TA's received such equal workload and only occasionally exchanged some of their groups to be graded. If this is really something that we want to support, it should be either Arie's or Mauricio's request imo.

Excluding groups that were previously assigned to a particular assignment is a reasonable extension to make.

TimvdLippe commented 7 years ago

Yes the TA's have the same budget, but at the start of the course Arie and us agreed that Liam and I performed more lab hour work and thus other TA's did more grading work. In the end, everyone does the same amount of hours for the whole course. This setup is necessary to be able to handle the large number of students efficiently, especially because we got a large set of new TA's.