search

castorini / anserini

Anserini is a Lucene toolkit for reproducible information retrieval research
http://anserini.io/
Apache License 2.0
1.03k stars 457 forks source link

Incorporate jtreceval directly into our repo #2247

Closed lintool closed 8 months ago

lintool commented 11 months ago

https://github.com/terrierteam/jtreceval

For example: https://github.com/castorini/anserini/blob/master/docs/regressions/regressions-msmarco-passage.md

We currently have:

tools/eval/trec_eval.9.0.4/trec_eval -c -m map tools/topics-and-qrels/qrels.msmarco-passage.dev-subset.txt runs/run.msmarco-passage.bm25-default.topics.msmarco-passage.dev-subset.txt

After integration, we should be able to execute:

target/appassembler/bin/trec_eval -c -m map tools/topics-and-qrels/qrels.msmarco-passage.dev-subset.txt runs/run.msmarco-passage.bm25-default.topics.msmarco-passage.dev-subset.txt

And with the fatjar, we can do:

java -cp anserini-0.22.2-SNAPSHOT-fatjar.jar trec_eval ...
Melissa1412 commented 11 months ago

I will pick it up.

lintool commented 10 months ago

@jasper-xian will work on this.

lintool commented 9 months ago

Initial integration in #2324 merged - we need additional refinements, so I'm leaving open for now.

lintool commented 8 months ago

Next improvement is described in #2391 - closing this issue.