Passage Re-ranking with BERT

[chullhwan-song]

https://arxiv.org/abs/1901.04085

[chullhwan-song]

Abstract

• query 기반 passage re-ranking
  • using BERT
• dataset
  • MS MARCO passage retrieval task
  • TREC-CAR dataset
• https://github.com/nyu-dl/dl4marco-bert

PASSAGE RE-RANKING WITH BERT

Task

• 3개의 main stage : question-answering pipeline
  1. BM25
     • a large number (for example, a thousand) of possibly relevant documents to a given question are retrieved from a corpus by a standard mechanism
  2. passage re-ranking
  3. Finally, the top ten or fifty of these documents will be the source for the candidate answers by an answer generation module.
     • 여기서는 당연히 passage re-ranking에 대해

       Method

• re-ranker는 query q 에 대한 후보 결과(BM25 결과..)에 대해 relevant score를 평가한다.
• 이때 re-ranker는 BERT를 적용.
• query : sentence A > 64 tokens
• the passage text (doc): sentence B
• MAX Sequence Length = 512( >= sentence A+sentence B)
• BERT는 이진 분류 모델
  • [CLS] vector를 input > single layer neural network > probability == relevant
• pre-trained BERT model and fine-tune
  • BM25결과( top-1,000 documents)에서의 J_pos는 quey와 relevance한 set , neg는 반대

실험