search

algoprog / Quin

An easy to use framework for large-scale fact-checking and question answering
MIT License
68 stars 7 forks source link

Questions about datasets used to train dense encoder models #5

Closed tdieu29 closed 3 years ago

tdieu29 commented 3 years ago

Hello, I read both of your papers that are linked in this repo and went through the code, and I'm a little bit confused about what datasets were used to train each of the dense encoder models that are listed in the README. From reading your papers, I'm assuming that (1) the dense encoder FC model was trained on NLI and Factual NLI datasets; (2) the dense encoder QA model was pretrained on NLI and Factual NLI and fine tuned on MSMARCO; and (3) the dense encoder M model was trained only on the Factual NLI+ dataset. Could you please confirm if this is correct? Thanks a lot!

algoprog commented 3 years ago

Correct for (1) and (2), however the multitask model (3) was trained on the concatenation of NLI, Factual NLI+ and MSMARCO. The code for training is not uploaded yet, but it depends on the sentence_transformers library.

tdieu29 commented 3 years ago

Thanks!