Check the bm25 top 10 is a subset of t5 top 10

ielab / GPT_Ranker

Use GPT-2/T5 to beat traditional LM

2 stars 0 forks source link

Closed hanglics closed 4 years ago

hanglics commented 4 years ago

when a relevant document retrieved by bm25 in top 10, is it also ranked by t5 in the top 10?

trec_eval -q -m recall_10 /qrels /res

do this for bm25 and t5

hanglics commented 4 years ago

R@10

Total in BM25: 2688
Total in T5: 3922
Only in BM25: 339
Only in T5: 1573
In BM25 And In T5: 2349

Reciprocal Rank@10

Intersection: 2436
T5 Better: 1082
BM25 Better: 736
Equal Better: 618