Support ARR (ACL, EMNLP, NAACL, etc)

[timbmg]

ACL Rolling Review (ARR) uses an Overall Assessment field to score papers. Could this be integrated?

[weigq]

@timbmg can you provide a link?

[timbmg]

EMNLP actually does not have Overall Assessment but only Excitement and Soundness scores, see here: https://openreview.net/forum?id=csBtifBXKo

For ACL and NAACL, unfortunately, the reviews are not public, so I can not give you a link where you can see the scores. However, I have copied a part of the /notes response that I suppose is relevant to extract the scores. (This is from a paper that I have reviewed, I have replaced all non-relevant content with ...).

{
    "notes": [
        {
            "id": "...",
            "forum": "...",
            "replyto": "...",
            "signatures": [
                "..."
            ],
            "nonreaders": [],
            "readers": [
                "..."
            ],
            "writers": [
                "..."
            ],
            "content": {
                "paper_summary": {
                    "value": "..."
                },
                "summary_of_strengths": {
                    "value": "..."
                },
                "summary_of_weaknesses": {
                    "value": "..."
                },
                "comments_suggestions_and_typos": {
                    "value": "..."
                },
                "confidence": {
                    "value": 3
                },
                "soundness": {
                    "value": 3.5
                },
                "overall_assessment": {
                    "value": 3.5
                },
                "best_paper": {
                    "value": "No"
                },
                "needs_ethics_review": {
                    "value": "No"
                },
                "reproducibility": {
                    "value": 4
                },
                "datasets": {
                    "value": 1
                },
                "software": {
                    "value": 1
                },
                "Knowledge_of_or_educated_guess_at_author_identity": {
                    "value": "No"
                },
                "Knowledge_of_paper": {
                    "value": "N/A, I do not know anything about the paper from outside sources"
                },
                "Knowledge_of_paper_source": {
                    "value": [
                        "N/A, I do not know anything about the paper from outside sources"
                    ]
                },
                "impact_of_knowledge_of_paper": {
                    "value": "N/A, I do not know anything about the paper from outside sources"
                }
            },
            "number": 1,
            "invitations": [
                "...",
                "..."
            ],
            "domain": "...",
            "tcdate": 1717996415000,
            "cdate": 1717996415000,
            "tmdate": 1718211332633,
            "mdate": 1718211332633,
            "license": "CC BY 4.0",
            "version": 2
        }
    ]
}