search

castorini / anserini

Anserini is a Lucene toolkit for reproducible information retrieval research
http://anserini.io/
Apache License 2.0
1.01k stars 444 forks source link

Add new flat regressions for MS MARCO v1 passage #2521

Closed lintool closed 2 months ago

lintool commented 3 months ago

For MS MARCO v1 dev, DL 19, DL 20.

Total of 36; here are the dev ones:

python src/main/python/run_regression.py --index --verify --search --regression msmarco-v1-passage.bge-base-en-v1.5.flat.cached > logs/log.msmarco-v1-passage.bge-base-en-v1.5.flat.cached.txt 2>&1 &
python src/main/python/run_regression.py --index --verify --search --regression msmarco-v1-passage.cohere-embed-english-v3.0.flat.cached > logs/log.msmarco-v1-passage.cohere-embed-english-v3.0.flat.cached.txt 2>&1 &
python src/main/python/run_regression.py --index --verify --search --regression msmarco-v1-passage.cos-dpr-distil.flat.cached > logs/log.msmarco-v1-passage.cos-dpr-distil.flat.cached.txt 2>&1 &
python src/main/python/run_regression.py --index --verify --search --regression msmarco-v1-passage.openai-ada2.flat.cached > logs/log.msmarco-v1-passage.openai-ada2.flat.cached.txt 2>&1 &

python src/main/python/run_regression.py --index --verify --search --regression msmarco-v1-passage.bge-base-en-v1.5.flat-int8.cached > logs/log.msmarco-v1-passage.bge-base-en-v1.5.flat-int8.cached.txt 2>&1 &
python src/main/python/run_regression.py --index --verify --search --regression msmarco-v1-passage.cohere-embed-english-v3.0.flat-int8.cached > logs/log.msmarco-v1-passage.cohere-embed-english-v3.0.flat-int8.cached.txt 2>&1 &
python src/main/python/run_regression.py --index --verify --search --regression msmarco-v1-passage.cos-dpr-distil.flat-int8.cached > logs/log.msmarco-v1-passage.cos-dpr-distil.flat-int8.cached.txt 2>&1 &
python src/main/python/run_regression.py --index --verify --search --regression msmarco-v1-passage.openai-ada2.flat-int8.cached > logs/log.msmarco-v1-passage.openai-ada2.flat-int8.cached.txt 2>&1 &

python src/main/python/run_regression.py --index --verify --search --regression msmarco-v1-passage.bge-base-en-v1.5.flat.onnx > logs/log.msmarco-v1-passage.bge-base-en-v1.5.flat.onnx.txt 2>&1 &
python src/main/python/run_regression.py --index --verify --search --regression msmarco-v1-passage.cos-dpr-distil.flat.onnx > logs/log.msmarco-v1-passage.cos-dpr-distil.flat.onnx.txt 2>&1 &

python src/main/python/run_regression.py --index --verify --search --regression msmarco-v1-passage.bge-base-en-v1.5.flat-int8.onnx > logs/log.msmarco-v1-passage.bge-base-en-v1.5.flat-int8.onnx.txt 2>&1 &
python src/main/python/run_regression.py --index --verify --search --regression msmarco-v1-passage.cos-dpr-distil.flat-int8.onnx > logs/log.msmarco-v1-passage.cos-dpr-distil.flat-int8.onnx.txt 2>&1 &

And for DL19 and DL20 also.

codecov[bot] commented 3 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 67.09%. Comparing base (59330e3) to head (ed93a9d).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #2521 +/- ## ========================================= Coverage 67.09% 67.09% Complexity 1472 1472 ========================================= Files 219 219 Lines 12628 12628 Branches 1526 1526 ========================================= Hits 8473 8473 Misses 3628 3628 Partials 527 527 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.