Got unsupported ScalarType BFloat16 in RerankingEvaluator.py

embeddings-benchmark / mteb

MTEB: Massive Text Embedding Benchmark

https://arxiv.org/abs/2210.07316

Apache License 2.0

1.91k stars 255 forks source link

Got unsupported ScalarType BFloat16 in RerankingEvaluator.py #1297

Open xujunrt opened 1 week ago

xujunrt commented 1 week ago

In the RerankingEvaluator.py, np.asarray does not support BFloat16. A forced conversion should be added to avoid similar errors. For example, you can first convert the Tensor to float16 and then call numpy_asarray.

isaac-chung commented 1 week ago

CC-ing @orionw and @KennethEnevoldsen on this.

@xujunrt it would be great if you could also share an example of how you're using the Reranking evaluator. Thanks!

KennethEnevoldsen commented 1 week ago

This sounds like it is related to #941

Agree with @xujunrt. This sounds like a problem we could have in more places as well.