beir-cellar beir issues

beir-cellar / beir

A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.

http://beir.ai

Apache License 2.0

1.48k stars 175 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Fix: mention proper metric used for retrieval

#177 kacperlukawski opened 1 week ago
0
can't reproduce DeepCT results with evaluate_deepct.py

#176 jeka-e opened 3 weeks ago
0
replacing `pytrec_eval` with `pytrec-eval-terrier`

#175 archersama opened 3 weeks ago
0
recommended pytrec-eval-terrier rather than pytrec-eval

#174 liyongkang123 opened 1 month ago
0
mt5-base model (unicamp-dl/mt5-base-en-msmarco) throws 'nan' outputs

#173 cramraj8 opened 1 month ago
0
Missing document from corpus ArguAna

#172 NourOM02 opened 1 month ago
0
How to Save Retrieved Documents in Retrieval Tasks

#171 AugustLHHHHHH opened 1 month ago
0
`retriever.retrieve(corpus, queries)` with `len(queries)==1` errors

#170 manestay opened 2 months ago
0
Why need check corpus_id != query_id in DenseRetrievalExactSearch.search()

#169 mengyao00 opened 2 months ago
1
TypeError: Cannot create a consistent method resolution order (MRO) for bases tqdm, tqdm_asyncio

#168 dinuransika closed 3 months ago
0
Evaluate "Offline" Models

#167 arendu opened 3 months ago
0
Rename Accuracy@k Metric to Success@k metric

#166 root-goksenin opened 3 months ago
0
Corpus encoding times for hotpotqa on A100 GPU

#165 jeyendranbalakrishnan opened 4 months ago
0
adding support for decoder-only LLMs e.g. E5-Mistral

#164 Krilecy closed 4 months ago
2
example file, train_msmarco_v2.py is not working

#163 bakingeol opened 4 months ago
0
Codespace ubiquitous acorn qpgj6564q7366vj

#162 konabuta closed 5 months ago
0
Malformed Chinese mMARCO dataset

#161 sissilab opened 6 months ago
0
Extras-require, base package without Faiss-CPU requirement

#160 raphaelsty opened 8 months ago
1
mMarco version

#159 carlesoctav opened 8 months ago
0
FIQA Empty Rows

#158 zanussbaum closed 9 months ago
3
Installation

#157 malinphy opened 9 months ago
3
Fine-tuned Cross-encoder scores are lower than ms-marco zero-shot scores ?

#156 cramraj8 opened 9 months ago
1
Support Multi-node evaluation

#155 NouamaneTazi opened 11 months ago
0
Fix `DenseRetrievalExactSearch` evaluation

#154 NouamaneTazi opened 11 months ago
0
Pull latest main branch into development

#153 thakur-nandan closed 11 months ago
0
Typing for retriever is too strict: implement abstract retriever base class

#152 jon-chuang opened 12 months ago
0
org.apache.lucene.search.BooleanQuery$TooManyClauses: maxClauseCount is set to 1024

#151 0ddAstronaut opened 1 year ago
0
Missing test.tsv in thakur/BEIR/datasets/nq-train.zip

#150 zhiyuanpeng closed 11 months ago
1
Redirect Anserini BM25 baselines in BEIR to Pyserini

#149 thakur-nandan opened 1 year ago
0
Merging the development branch with the latest main branch

#148 thakur-nandan closed 11 months ago
2
Reproduce Signal-1M

#147 Dundalia closed 1 year ago
2
evaluate_anserini_bm25.py retrieves 1000 documents for each query no matter which k I set in payload

#146 zhiyuanpeng opened 1 year ago
2
Don't rebuild bm25 pyserini index every time when running the script

#145 zhiyuanpeng closed 1 year ago
0
adding a simple implementation of ColBERT

#144 jjmachan opened 1 year ago
6
Can't reproduce BM25 baselines

#143 zhiyuanpeng closed 1 year ago
7
Update README.md

#142 eltociear closed 11 months ago
0
Training script/configuration of T5 generator for GenQ

#141 jihyukkim-nlp opened 1 year ago
2
AssertionError: Elastic-Search Window too large, Max-Size = 10000

#140 zuliani99 opened 1 year ago
0
[Question] Using T5 as a retriveal model in BeIR

#139 rahmanidashti opened 1 year ago
2
No module named 'tensorflow_text'

#138 sadanyh opened 1 year ago
3
Loosen Elasticsearch requirement

#137 stephantul opened 1 year ago
1
Development

#136 ashokrajab closed 1 year ago
0
Allow option to disable progress bar when using `GenericDataLoader`

#135 xhluca opened 1 year ago
2
evaluate_sbert_multi_gpu - metrics.compute() unable to read cache file

#134 ashokrajab opened 1 year ago
1
Fix BioASQ Avg Q/D according to Paper

#133 timsteuer closed 11 months ago
0
Can't download data

#132 TahaBinhuraib closed 1 year ago
2
Can't explain low results for some models

#131 NebelAI opened 1 year ago
1
Adding the ANTIQUE dataset to BEIR

#130 heliah opened 1 year ago
4
Reproducability for all-minilm-l6-v2

#129 Manikseam opened 1 year ago
1
Dataset Characteristics Report on `data_loader` Function

#128 rahmanidashti opened 1 year ago
0