BUG：bce-reranker-base_v1 corpus tensor 超长

[linhao622]

Describe the bug

使用bce-reranker-base_v1模型时，corpus超长报错

To Reproduce

test_rerank.py

from xinference.client import Client client = Client("http://localhost:9997")

先下载再本地注册模型，模型名称带个local后缀

model = client.get_model('bce-reranker-base_v1-local')

model = client.get_model('bge-reranker-v2-m3-local')

query = "A man is eating pasta." tmp = 'A man is eating food.A man is eating a piece of bread.The girl is carrying a baby.A man is riding a horse.A woman is playing violin.'

tmp的字符长度是133

corpus = [tmp*300] print(model.rerank(corpus, query))

(3.10.13) tsuyoshidomoto@ymy:~/1506/py$ python test_rerank.py Traceback (most recent call last): File "/Users/tsuyoshidomoto/1506/py/test_rerank.py", line 20, in print(model.rerank(corpus, query)) File "/Users/tsuyoshidomoto/.pyenv/versions/3.10.13/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 181, in rerank raise RuntimeError( RuntimeError: Failed to rerank documents, detail: [address=0.0.0.0:60205, pid=59225] The expanded size of the tensor (11410) must match the existing size (514) at non-singleton dimension 1. Target sizes: [1, 11410]. Tensor sizes: [1, 514]

Expected behavior

bge-reranker-v2-m3的max_position_embeddings是8194 bce-reranker-base_v1的是514 但是corpus=tmp*300时，总tensor长度是11410，超过两者最大值，bge的模型可以出结果，bce的就报错

Additional context

经过测试，使用bce-rerank模型时，只要corpus里的文本总tensor长度超过514就会报错，

[1402564807]

我也有这个错误，求解决办法

[github-actions[bot]]

This issue is stale because it has been open for 7 days with no activity.

[zhangever]

我这边也遇到同样的问题，希望能解决一下。