search

netease-youdao / BCEmbedding

Netease Youdao's open-source embedding and reranker models for RAG products.
Apache License 2.0
1.3k stars 85 forks source link

Input validation error: `inputs` must have less than 512 tokens. Given: 1009 #65

Open nikhilcms opened 2 months ago

nikhilcms commented 2 months ago

Hi, using huggingface/text-embeddings-inference I have deploy bce-reranker-base_v1 model.

In the info endpoint and in your provided example I can see you are setting max_length=512

could you confirm this model support only 512 tokens only or Is there any way to process long text without truncate ?

shenlei1020 commented 1 month ago

Thank you for your interests!

  1. We propose our bce-reranker-base_v1 for reranking long passages (each passage < 32k tokens) by our python package BCEmbedding. You can install it simply by pip install BCEmbedding.
  2. The usage for reranking long can be checked in "https://github.com/netease-youdao/BCEmbedding?tab=readme-ov-file#1-based-on-bcembedding". We must mention that our bce-reranker-base_v1 just supports max length for 512, and the method for reranking long passages is open-source (see "NOTE" in the url above), which is a good balance between efficiency and effectivity (also adopted by other projects).
  3. If you want to build a server in production, you can refer to "https://github.com/netease-youdao/BCEmbedding/blob/master/BCEmbedding/models/reranker.py#L101" to add this feature in your raw code.