search

hotpotqa / hotpot

Apache License 2.0
444 stars 75 forks source link

DocQA or improved BiDAF? #8

Closed valsworthen closed 5 years ago

valsworthen commented 5 years ago

Hello,

this may be a dumb question but from the code I have the feeling that the model looks more like BiDAF than DocQA. DocQA is an improved BiDAF with a retrieval system to select a single paragraph on which to apply BiDAF. However if I'm not mistaken the preprocessing script joins all the Hotpot paragraph instead of selecting one. The only retrieval is in the creation of the fullwiki setup where 10 paragraphs are selected.

Sure the model contains improvements compared to BiDAF (self-attention, 3-way classifier, no highway layer) but not the main particularity of DocQA which is the retriever. Yet in the paper states that the authors implemented DocQA.

Can you confirm that the code indeed merges all the paragraphs?

Thanks.

kimiyoung commented 5 years ago

There are two stages. In the retrieval stage, we used bigram tfidf (which is also used in DrQA). In the comprehension stage, we concatenate all (retrieved) paragraphs and build a model to predict the answer.

valsworthen commented 5 years ago

Ok so if I understand, the retrieval stage is performed on all of wikipedia and returns the 10 paragraphs that form each example? And from DocQA by Clark & Gardner you use the merge approach which consists in concatenating these 10 paragraphs?

kimiyoung commented 5 years ago

Yes that's right. In terms of possible research directions, one might be interested in improving either the retrieval methods, the comprehension model, or a better pipeline.