cmdowney88
commented
3 years ago @fxiafxia and I have noticed that quite a lot of the reviewer-paper similarity scores seem to be zero. Judging from the diagnostic printouts of the system, we're guessing this is because a large proportion of the reviewers don't put in a Semantic Scholar ID, put it in incorrectly (such that it can't be parsed), try to put in some other link (like Google Scholar), or simply don't have a Semantic Scholar ID yet. This seems like a substantial issue. Perhaps it can be partly solved with emphasis through the reviewer-recruiting process that they should put in their ID if they have one, and format it correctly. However, lots of people also have difficulty with Semantic Scholar (i.e., they are indexed incorrectly based on things like hyphenated names). Should there be some sort of workaround based on, say, keywords that reviewers and authors can put in to be matched?
Currently semantic scholar integer IDs are supported by the code, but this only works if they already exist in softconf. It's not 100% clear that all reviewers will have their IDs entered in softconf by the time that the matching software is run, so some consideration of how to do things in that case is required.
It might be a good idea to create a script to try to infer missing semantic scholar ID by matching against the ACL anthology, but this wouldn't work if some sort of disambiguation needs to be done.
Also, how to get more papers from semantic scholar based on semantic scholar IDs is not documented.