Autoassign does not always respect (what should be) hard constraints

[cascremers]

Expected behavior of auto-assign:

• The mode "Ensure each selected paper has at least N [...] review(s)" should be a hard constraint, not a soft one.

Observed behavior:

• Instance in which constraint was not met by assignment, even though it would clearly be possible.

In a multi-track setup, we have observed that the auto-assign (performed over all tracks at once) did not assign the requested 2 reviewers per paper, even when this was explicitly selected. Instead, some papers ended up with 1 reviewer, even though it was clear this could be easily resolved. In particular, using individual auto-assign actions per track worked as expected.

Conjecture: this may be caused by imbalance between the average load (reviews per reviewer) between tracks, in which case the autoassign algorithm does not seem to explore all possible options.

In our concrete example, we had one track with 1-2 reviews per reviewer, and another with 3-4 reviews per reviewer, i.e., some reviewers would get 1 review, others 4, in any optimal assignment because of the track setup. However, the autoassign would not give any reviewer 4 reviews, but only at most 3. Thus, the track with >3 average had some papers with only 1 reviewer. Using auto-assign on each track individually worked as expected.

[kohler]

1. Is this on HotCRP.com? Which conference?
2. Do you have “bad pairs” selected?
3. Do all admins have the same track permissions?

Your conjecture doesn’t seem likely to me given the implementation.

Message ID: @.***>

[cascremers]

A1. This is (was) on ccs2022a.hotcrp.com A2. No, there were no bad pairs selected. A3. Track chairs have permissions for their own track, not beyond; but these didn't play a role here I think. I did both types of assignment in my admin role (pc co chair, not track specific).