Accuracy get from running the code is far less than in the paper

cooelf / SemBERT

Semantics-aware BERT for Language Understanding (AAAI 2020)

https://arxiv.org/abs/1909.02209

MIT License

285 stars 55 forks source link

Closed amyzhw closed 4 years ago

amyzhw commented 4 years ago

Hi, The acc on MNLI is reported as 84.0 for SemBERT_base. However, I can only get 33.3 from running the code without any modification.

Could you let me know if I missed anything?

Thanks much, Amy

cooelf commented 4 years ago

Please check if you used the sample data in the folder glue_data/MNLI, which is just used as example.

You may annotate the semantic role labels using the tools following the instructions (especially for your own data)