Open neubig opened 2 years ago
Suggestion from Avi at IBM, so I guess we should focus on PrimeQA's TyDiQA:
We’ve been focusing on TyDI for sometime now: the model’s available here too: https://huggingface.co/PrimeQA/tydiqa-primary-task-xlm-roberta-large. So if you want to do the same expts with TyDI and use PrimeQA you can totally get the system outputs by running decoding. It should be pretty straightforward.
There is a nice PrimeQA framework developed by IBM, and it seems that it should make it relatively easy to generate open-domain QA results from multiple SOTA models.
It would be nice if we could look into these results, analyze them with explainaboard, and see if there are any ways we could use them to improve our analysis of Open Domain QA models.
In order to do this, we'd need to: