search

beir-cellar / beir

A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.
http://beir.ai
Apache License 2.0
1.55k stars 186 forks source link

Custom multilingual issues #32

Closed Graduo closed 3 years ago

Graduo commented 3 years ago

Hi, thanks for your awesome work! Dose this framework support Chinese ? How can I use it in my own Chinese dataset (sparse ,dense ...),I mean that can I use my own tokenizer?

thanks