embeddings-benchmark arena issues

embeddings-benchmark / arena

Code for the MTEB Arena

https://hf.co/spaces/mteb/arena

13 stars 5 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Breakdown Across Corpus

#41 zanussbaum opened 2 weeks ago
0
Diff between retrieved documents

#40 Muennighoff opened 3 weeks ago
0
Top1 vs Top2 of same model

#39 Muennighoff opened 3 weeks ago
0
Fix HF repo error when loading bm25 index

#38 isaac-chung opened 4 weeks ago
0
WIP: feat: Add ColBERT

#37 bclavie opened 1 month ago
0
Do more to prevent ties

#36 KennethEnevoldsen opened 1 month ago
2
Add normalization kwarg for encode

#35 violenil closed 1 week ago
0
Normalization during retrieval scores computation

#34 violenil opened 1 month ago
2
huggingface_hub.errors.HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: 'mteb/index_stackexchange_### model a: bm25'.

#33 Muennighoff opened 1 month ago
3
Support for Retrieval-only models (ColBERT?)

#32 bclavie opened 1 month ago
7
Check "both are bad" / "tie" votes & consider displaying top-K

#31 Muennighoff opened 1 month ago
2
Why is Alibaba-NLP/gte-Qwen2-7B-instruct so bad?

#30 Muennighoff closed 1 month ago
2
Connect leaderboard with crowdsourcing platform

#29 Muennighoff opened 1 month ago
0
Skip responses when passages are the same?

#28 orionw opened 1 month ago
1
Use the CI in determining ranking numbers

#27 orionw opened 1 month ago
0
fix: nomic embed prefix, v1 to arena

#26 zanussbaum closed 1 month ago
1
Turn MTEB-Arena-logs into HF dataset?

#25 Muennighoff opened 1 month ago
4
KeyError: '### Model A: voyage-multilingual-2'

#24 Muennighoff opened 1 month ago
0
Classification tab

#23 Muennighoff opened 1 month ago
1
Make Clustering more intuitive

#22 Muennighoff opened 1 month ago
0
Nomic Models Not Using Query (or Document) Instructions

#21 zanussbaum closed 1 month ago
9
Track time people take to vote

#20 Muennighoff opened 1 month ago
3
Add clustering samples

#19 orionw closed 1 month ago
1
Allow non-ascii characters in jsonl

#18 Muennighoff closed 1 month ago
4
Add leaderboard categories

#17 Muennighoff opened 1 month ago
0
Corpus improvements

#16 Muennighoff opened 1 month ago
0
Wikipedia corpus tables ignored

#15 Muennighoff opened 1 month ago
1
Add personal leaderboard

#14 Muennighoff opened 1 month ago
8
Separate Side-by-Side results and Battle results; Update Arxiv text

#13 orionw closed 2 months ago
5
Final launch TODOs

#12 Muennighoff closed 1 month ago
54
[WIP] Add bm25s

#11 isaac-chung closed 2 months ago
1
Remove Git LFS from repo

#10 orionw opened 2 months ago
0
WIP: Create indices

#9 Muennighoff closed 2 months ago
4
Add chunking for arxiv and stackexchange

#8 orionw closed 2 months ago
0
Wikipedia parsing and chunking

#7 orionw closed 2 months ago
3
Models to support for launch

#6 Muennighoff closed 1 month ago
14
Allow selecting different corpora in Retrieval

#5 Muennighoff closed 1 month ago
27
Add share button to Clustering arena

#4 Muennighoff opened 2 months ago
0
Add share button to STS arena

#3 Muennighoff opened 2 months ago
0
Add GCP Vertex search index

#2 isaac-chung closed 2 months ago
32
General feedback

#1 KennethEnevoldsen closed 1 month ago
16