Improve retriever model

cdqa-suite / cdQA

⛔ [NOT MAINTAINED] An End-To-End Closed Domain Question Answering System.

https://cdqa-suite.github.io/cdQA-website/

Apache License 2.0

614 stars 191 forks source link

Improve retriever model #158

Closed fmikaelian closed 5 years ago

andrelmfarias commented 5 years ago

Some ideas of improvement:

Implement a Retriever with Doc2vec (https://radimrehurek.com/gensim/models/doc2vec.html)
Implement a Retriever with passage re-ranking with BERT (https://arxiv.org/abs/1901.04085)
Retrieve paragraphs instead of articles

alex-movila commented 5 years ago

"passage re-ranking with BERT" this may be too slow since it uses BERT "Retrieve paragraphs instead of articles" - This would be great since I do not need multiple articles.

alex-movila commented 5 years ago

Here is some successful implementation. They use an ensemble of different algorithms combined with logistic regression. Could be useful for inspiration: https://allenai.org/aristo/

andrelmfarias commented 5 years ago

Hi @alex-movila thanks for indicating this project. However, as far as I know, the Aristo model is only able to answer questions which have a small list of (usually 4 or 5) possible answers.

Our system is able to find answers for open questions, using a database of documents where it can find the answer.

Unfortunately, it's not useful for us...