I ran the script for relation extraction in Chemprot. It looks like the accuracy includes all relations available in Chemprot, while the original publication of Chemprot and other relevant publications like bioBERT evaluated over only 5 relations ( specifically, AGONIST, ANTAGONIST, SUBSTRATE, UPREGULATOR and DOWNREGULATOR ). So I wonder whether the SciBERT paper reported the accuracy (83.64) over these 5 relations, or all relations?
I ran the script for relation extraction in Chemprot. It looks like the accuracy includes all relations available in Chemprot, while the original publication of Chemprot and other relevant publications like bioBERT evaluated over only 5 relations ( specifically, AGONIST, ANTAGONIST, SUBSTRATE, UPREGULATOR and DOWNREGULATOR ). So I wonder whether the SciBERT paper reported the accuracy (83.64) over these 5 relations, or all relations?