embeddings-benchmark mteb issues

embeddings-benchmark / mteb

MTEB: Massive Text Embedding Benchmark

https://arxiv.org/abs/2210.07316

Apache License 2.0

1.61k stars 210 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Update points.md

#1010 mariyahendriksen opened 14 hours ago
0
Paper writing: Authors + appendix author contributions [missing]

#1009 KennethEnevoldsen opened 22 hours ago
1
Paper writing: Appendix B 1-2 [needs review]

#1008 KennethEnevoldsen opened 23 hours ago
0
Paper Writing: Speeding up the benchmark [waiting for retrieval]

#1007 KennethEnevoldsen opened 23 hours ago
0
Paper Writing: Methods - The Open-source Effort [in need of review]

#1006 KennethEnevoldsen opened 23 hours ago
0
Paper Writing: Related works [needs review]

#1005 KennethEnevoldsen opened 23 hours ago
1
Paper Writing: Introduction [in need of review]

#1004 KennethEnevoldsen opened 23 hours ago
2
Added benchmark construction

#1003 KennethEnevoldsen opened 1 day ago
2
add SFR embed model

#1002 isaac-chung opened 1 day ago
0
install only ruff for lint workflow

#1001 Samoed opened 1 day ago
1
Add more russian models

#1000 Samoed opened 1 day ago
2
extracting embedding from other than CLS token during eval mteb

#999 riyajatar37003 opened 1 day ago
0
The Swahili Classification Task

#998 msamwelmollel opened 1 day ago
0
LLM2Vec inference error

#997 AlexeyVatolin opened 1 day ago
2
docs: Added result tables

#996 KennethEnevoldsen opened 1 day ago
0
Added utility function for downloading and loading models results

#995 KennethEnevoldsen closed 1 day ago
0
Update DanFever to use avoid using overwrite=True and ignore identical ids

#994 KennethEnevoldsen closed 2 days ago
1
fix: convert scores in meta to be in 0-100 instead of 0-1

#993 KennethEnevoldsen closed 2 days ago
0
calculate_metadata_metrics on MSMARCOv2 goes OOM

#992 isaac-chung opened 2 days ago
0
Discrepancy in Score Ranges Between Metadata and Leaderboard in MTEB

#991 omarnj-lab closed 2 days ago
10
Adding a BM25 baseline?

#990 malteos opened 3 days ago
2
Cannot load e5

#989 ngocbh closed 2 days ago
1
Update numbers in retrieval statistics

#988 orionw closed 2 days ago
8
Add Nomic and Multilingual Cohere

#987 KennethEnevoldsen closed 2 days ago
0
Add BGE large en v1.5

#986 isaac-chung closed 2 days ago
0
Update Retrieval Statistics Checker

#985 orionw closed 3 days ago
6
Option to remove cached dataset files on large runs

#984 isaac-chung opened 4 days ago
2
GPUtil is optional but required by default

#983 imenelydiaker closed 5 days ago
1
Move speedtask imports

#982 Samoed closed 5 days ago
0
Fix GritLM Retrieval instructions

#981 Muennighoff closed 4 days ago
5
Voyage bclavie/mmarco-japanese-hard-negatives

#980 jordiclive closed 3 days ago
0
add points

#979 jordiclive closed 5 days ago
0
Incorrect logic when retrieval top-k

#978 thangylvp opened 6 days ago
1
fix: add max_fraction_of_documents_to_embed to clustering datasets with `max_document_to_embed`

#977 Samoed closed 5 days ago
1
Fix GritLM instructions

#976 Muennighoff closed 5 days ago
0
create-table fails

#975 isaac-chung closed 6 days ago
2
fix: ensure that results from parallel datasets are formatted correctly

#974 KennethEnevoldsen closed 3 days ago
4
GritLM not using instructions

#973 Muennighoff closed 5 days ago
1
[WIP] RAR-b second PR & task instruction for IR tasks in general

#972 gowitheflow-1998 closed 1 day ago
14
Add support for the Bright benchmark

#971 swj0419 opened 1 week ago
1
Fix GritLM kwarg

#970 Muennighoff closed 1 week ago
0
KeyError: 'default'

#969 Muennighoff closed 3 days ago
3
BibleNLPBitextMining file missing

#968 Muennighoff closed 1 week ago
5
fix: Add Speed tasks for CPU and GPU & system info

#967 isaac-chung closed 1 week ago
1
docs: fix cli commands

#966 Samoed closed 1 week ago
0
fix: bug in calculate_metadata_metrics for some retrieval datasets

#965 isaac-chung closed 1 week ago
1
Some retrieval datasets don't calculate metadata correctly

#964 orionw opened 1 week ago
4
Update annotations for PawsX, Opusparcus and SummEval

#963 artemsnegirev closed 1 week ago
1
Add baseline models for Russian

#962 artemsnegirev closed 1 week ago
0
Update points.md

#961 ABorghini closed 1 week ago
0