embeddings-benchmark mteb issues

embeddings-benchmark / mteb

MTEB: Massive Text Embedding Benchmark

https://arxiv.org/abs/2210.07316

Apache License 2.0

1.97k stars 274 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

[mieb] Fix Moco model on CIFAR10Clustering

#1487 isaac-chung closed 4 hours ago
0
docs: Add lang family mapping and map to task table

#1486 isaac-chung opened 5 hours ago
0
Add Language Families

#1485 isaac-chung opened 7 hours ago
0
fix: Fix `BrightRetrieval` calculate stats

#1484 Samoed opened 22 hours ago
0
calulate_metadata_metrics fails with BrightRetrieval task

#1483 imenelydiaker opened 1 day ago
1
1475 add descriptive stats to all tasks v2

#1482 dokato opened 1 day ago
2
Add text-embedding-005

#1480 emerzon opened 1 day ago
0
[mieb] nyu-visionx/moco-v3-vit-b on CIFAR10Clustering: Unexpected type <class 'list'>

#1479 Muennighoff closed 4 hours ago
1
multi vector models?

#1478 raghavlite closed 1 day ago
1
add citations to ModelMeta

#1477 Samoed closed 1 day ago
0
Add descriptive stats to mising tasks and add number of qrels

#1476 imenelydiaker closed 1 day ago
5
Add `descriptive_stats` to all tasks

#1475 Samoed opened 3 days ago
11
add google/siglip-so400m-patch16-256-i18n

#1474 SaitejaUtpala opened 3 days ago
1
feat: Multilingual retrieval loader

#1473 Samoed closed 3 days ago
2
[mieb] run script for missing results

#1472 gowitheflow-1998 closed 3 days ago
0
fix: pin datasets version <3.0.0

#1471 Napuh closed 3 days ago
0
fix: pinned datasets to <3.0.0

#1470 Napuh closed 3 days ago
4
fix: swap touche2020 to maintain compatibility

#1469 isaac-chung closed 4 days ago
1
docs: Add sum per language for task counts

#1468 isaac-chung closed 4 days ago
0
Issues with benchmarks.py

#1467 Muennighoff closed 2 days ago
5
Add decriptive stat almost to all datasets

#1466 Samoed closed 4 days ago
2
Can't load some datasets because of datasets 3.0

#1465 Samoed opened 5 days ago
2
Add NanoBEIR

#1464 Muennighoff opened 6 days ago
1
Importing mteb is slow

#1463 Muennighoff opened 6 days ago
7
[mieb] Fix VLM2vec dtype

#1462 isaac-chung closed 3 days ago
0
WIP: Polishing up leaderboard UI

#1461 x-tabdeveloping closed 1 week ago
1
fix: loading pre 1.11.0

#1460 Samoed closed 1 week ago
3
feat: add CUREv1 retrieval dataset

#1459 dbuades closed 22 hours ago
4
[mieb] add support for LLM2CLIP

#1458 SaitejaUtpala opened 1 week ago
0
[mieb] TIGER-Lab/VLM2Vec-Full torch.cat(): expected a non-empty list of Tensors

#1457 Muennighoff closed 3 days ago
2
[mieb] TIGER-Lab/VLM2Vec-LoRA fails with dtype issue

#1456 Muennighoff closed 1 week ago
1
[mieb] royokong/e5-v on EDIST2ITRetrieval 'NoneType' object has no attribute 'dtype'

#1455 Muennighoff opened 1 week ago
0
fix: Merge main into v2

#1454 Samoed closed 1 week ago
0
fix: Fix load external results with `None` mteb_version

#1453 Samoed closed 1 week ago
0
fix: publish

#1452 x-tabdeveloping closed 1 week ago
0
Fixed task loading

#1451 x-tabdeveloping closed 1 week ago
0
Fix: Made data parsing in the leaderboard figure more robust

#1450 x-tabdeveloping closed 1 week ago
1
[MIEB] Change Flickr30k to test split

#1449 Jamie-Stirling closed 1 week ago
0
fix: update task metadata to allow for null

#1448 KennethEnevoldsen closed 1 week ago
0
fix: Unsure TaskResults can handle runtime and version being unspecified

#1447 KennethEnevoldsen closed 1 week ago
0
[mieb] add support for voyage-multimodal-3:

#1446 Muennighoff opened 1 week ago
0
"KeyError: 'document' not found and no similar keys were found.

#1445 LeMoussel opened 1 week ago
5
fix: remove NaN handling

#1444 KennethEnevoldsen closed 1 week ago
3
Leaderboard: Add radar chart

#1443 KennethEnevoldsen opened 1 week ago
0
Model metas: E5 instruct not listed as sbert compatible

#1442 x-tabdeveloping opened 1 week ago
0
Leaderboard: Fixed code benchmarks

#1441 x-tabdeveloping closed 1 week ago
4
Leaderboard: Add query parameters to link

#1440 x-tabdeveloping opened 1 week ago
0
Leaderboard: Fix code benchmarks

#1439 x-tabdeveloping closed 1 week ago
3
fix: Count unique texts, data leaks in calculate metrics

#1438 Samoed closed 1 week ago
4
Leaderboard 2.0: added performance x n_parameters plot + more benchmark info

#1437 x-tabdeveloping closed 1 week ago
8