search

xorbitsai / inference

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
https://inference.readthedocs.io
Apache License 2.0
4.7k stars 368 forks source link

Xinference 只提供bge的rerank模型？？？ #1398

Closed liangpn closed 4 days ago

liangpn commented 4 months ago

fastgpt 调用 xinference rerank接口报错 fastgpt的配置 image

使用bce的reank会有下面的错误 69cb08bb59c01b9eea328d3d2a898f6 61622e47e94db52755c596b9f40dac0 使用bge的reank则是正常 f2a56b93a0dba4f31eaad604966129f cc0b1ab03cd32142d7743ba5e109f8f

但是直接curl 调用xinference 的rerank 接口是返回正常 image

qinxuye commented 4 months ago

Xinference 如果 curl 没问题，感觉是 FastGPT 那边的问题。

liangpn commented 4 months ago

Xinference 如果 curl 没问题，感觉是 FastGPT 那边的问题。 是的，curl 是没问题的，fastgpt那边我也提了issue.

NaiveYan commented 4 months ago

dify上复现了类似的报错。 dify调用部署在0.10.1和0.10.3版本xinference上的bce reranker模型都会报类似的错误，但0.10.2.post1版本可以正常使用。

codingl2k1 commented 4 months ago

但是只有0.10.2post1用的是FlagEmbedding推理的，0.10.3又改回了sentence transformers推理因为很多用户说FlagEmbedding出错。

NaiveYan commented 3 months ago

v0.11.0上问题依然存在。我检查了一下具体的http request，当documents为较长的代码片段（1500~1700字符）时，使用bce模型会报错，使用bge模型可以正常rerank xinference上运行的模型如图 Screenshot 2024-05-13 at 16-29-17 Xinference 使用bce模型会报错 bce bge模型可以正常使用 bge 具体post request如下 bce.request.json xinference错误日志如下 xinference.error.log

github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 7 days with no activity.

github-actions[bot] commented 3 weeks ago

This issue was closed because it has been inactive for 5 days since being marked as stale.