Closed fabiopetroni closed 4 years ago
Implementation of evidence set aware recall. A model should produce a list of page ids. The core idea is to group page ids that belongs to the same evidence set (consider them as 1 in the rank). Pages that are not evidence counts as 1 as well.
test:
python kilt/eval_retrieval.py /checkpoint/fabiopetroni/KILT/predictions/DPR_nicola/nq-dev-kilt.jsonl /checkpoint/fabiopetroni/KILT/datasets/nq-dev-kilt.jsonl
Implementation of evidence set aware recall. A model should produce a list of page ids. The core idea is to group page ids that belongs to the same evidence set (consider them as 1 in the rank). Pages that are not evidence counts as 1 as well.
test:
python kilt/eval_retrieval.py /checkpoint/fabiopetroni/KILT/predictions/DPR_nicola/nq-dev-kilt.jsonl /checkpoint/fabiopetroni/KILT/datasets/nq-dev-kilt.jsonl