Closed timhartill closed 2 years ago
Hi, If I train my own question encoder using train_mhop.py and your hpqa train files and then run eval_mhop_retrieval.py against the new encoder my results are all 0:
Evaluating 7405 samples... Avg PR: 0.0 Avg P-EM: 0.0 Avg 1-Recall: 0.0 Path Recall: 0.0 comparison Questions num: 1487 Avg PR: 0.0 Avg P-EM: 0.0 Avg 1-Recall: 0.0 Path Recall: 0.0 bridge Questions num: 5918 Avg PR: 0.0 Avg P-EM: 0.0 Avg 1-Recall: 0.0 Path Recall: 0.0
Ah..I see now I need to run encode_corpus.py to generate new passage embeddings then run eval...
Hi, If I train my own question encoder using train_mhop.py and your hpqa train files and then run eval_mhop_retrieval.py against the new encoder my results are all 0:
Evaluating 7405 samples... Avg PR: 0.0 Avg P-EM: 0.0 Avg 1-Recall: 0.0 Path Recall: 0.0 comparison Questions num: 1487 Avg PR: 0.0 Avg P-EM: 0.0 Avg 1-Recall: 0.0 Path Recall: 0.0 bridge Questions num: 5918 Avg PR: 0.0 Avg P-EM: 0.0 Avg 1-Recall: 0.0 Path Recall: 0.0
Ah..I see now I need to run encode_corpus.py to generate new passage embeddings then run eval...