How can I evaluate COPA dataset? - Githubissues

microsoft / DeBERTa

The implementation of DeBERTa

MIT License

1.91k stars 215 forks source link

How can I evaluate COPA dataset? #150

Open KwanghyeonLee opened 5 months ago

KwanghyeonLee commented 5 months ago

Hi, I'm looking for evaluating COPA dataset in SUPER_GLUE benchmark for my project. As I run the code before, it din't work with assertion error.

AssertionError: COPA is not registed. Valid tasks ['sts-b', 'rte', 'mrpc', 'qnli', 'cola', 'sst-2', 'qqp', 'mnli', 'anli', 'ner', 'race']

How can I evaluate COPA dataset in this github repository?