Open jennybc opened 5 years ago
Thank you so much @jennybc for this suggestion! We used the Latin Square approach within the peer_roster_create()
function (in PR #66).
latin_square = function(j, n) {
i <- seq_len(n)
(((i - 1) + (j - 1)) %% n) + 1
}
j = sample(2:length(user), n_rev)
res = purrr::map(j, ~ latin_square(.x, length(user)))
As promised in Slack, here's code I used for assigning peer reviewers. Apparently I passed this along to other instructors in UBC's MDS program at some point, so here's my exposition. Basically we just use columns from a Latin square.
Functions needed to assign reviewers
Worked example.
Let n be the number of students and m be the number of peer reviews given/received.
self
defaults to TRUE, which means the first column will be 1:n. Set toFALSE
to suppress that.Test the constraints.
Now you will use the columns of
hw_assign
to index into student names. If you want more exposition, read on.Exposition
Peer review assignments must obey these constraints:
This is why I hold the peer review assignments for one homework as a matrix with n rows, where n is the number of students, and m + 1 columns, where m is the number of peer reviews given/received. For example, if m = 2:
We meet the constraints when columns 2 and 3 hold permutations of 1:n and each row holds 3 distinct values.
I used to generate the assignments randomly, but I recently realized there’s a deterministic solution. Consider an n x n Latin square with 1:n as the first column. Pick two of the other columns at random. You’re DONE.
Just FYI: One can read about random generation of latin squares here: http://math.stackexchange.com/questions/63131/generate-random-latin-squares. But note we don’t need to do this, just picking valid ones from the Latin square is good enough. I never promised a uniform distribution over all possible peer review assignments! However, if the set of students if fixed, like it is in MDS, you might want to do a bunch of peer review assignments at once, if you’re worried about re-using the same column by chance.
Explicit examples of the logic
Here’s the entire Latin square.
Here’s how to generate just the i,j-th entry.
Here’s how to generate j-th column, which is what we do above in the main function.