trevorcohn
commented
3 years ago The "Track" field was used to record the track assignment of the paper. Subject Area was the author's choice of track, while Track was our choice, after looking at COIs, and having a back-and-forth with SACs about topical fit. I don't recall if we used the Secondary Subject Area, definitely not automatically, but I suspect we would have when manually moving papers.
The Moving-To-COI-Track hack was to allow each COI paper to be assigned a AC in its home track, then reviewers in its home track. The AC would double check the reviewer assignment. Then we move it to COI track, and it gets a new AC, but carries the reviewers with it. This seemed to work fairly smoothly, but did require this hack.
The file looks like this:
Submission ID,Move To,COI with Heng Ji,R1,R2,R3 12345,Multidisciplinary and AC COI,#N/A,Name (username),Name (username),Name (username) where Heng Ji was the sole SAC for the COI track, and this entry would have a number when there was a COI. I think this is not relevant for assignment.
You will need to be granted access to https://github.com/acl-org/reviewer-coi-detection, as this is where the COI detection is done.
I think the author field was a historical thing, and there is some redundancy carrying over from Graham et al's original code.
And yes, requirements is probably missing a few things, sorry!
Trying to run this code for NAACL 2021, got some questions:
Trackfield, is it different fromSubject Area? IsSecondary Subject Areaever used? https://github.com/acl-org/reviewer-paper-matching/blob/674734a566ed7dfaa738b9ee7f20048fe2ff9730/softconf_extract.py#L90suggest_reviewers.pywith a filteredreviewers.jsonlfile?suggest_ac_reviewers_by_track.pyseems to have a hard-coded link toMoving-to-COI-Track.csv: https://github.com/acl-org/reviewer-paper-matching/blob/674734a566ed7dfaa738b9ee7f20048fe2ff9730/suggest_ac_reviewers_by_track.py#L276 Could this be added toREADME?Moving-to-COI-Track.csvand what is the "Move to" column? https://github.com/acl-org/reviewer-paper-matching/blob/674734a566ed7dfaa738b9ee7f20048fe2ff9730/suggest_ac_reviewers_by_track.py#L281suggest_ac_reviewers_by_track.py. Could this be clarified inREADME?calc_reviewer_db_mappingdoes not haveauthor_colargument anymore? Is it safe to remove it from the function calls insuggest_reviewers.py? https://github.com/acl-org/reviewer-paper-matching/blob/674734a566ed7dfaa738b9ee7f20048fe2ff9730/suggest_reviewers.py#L192 Also notice that the documentation for this field still exists: https://github.com/acl-org/reviewer-paper-matching/blob/674734a566ed7dfaa738b9ee7f20048fe2ff9730/suggest_utils.py#L20requirements.txtseems to be missing a few things.