search

xorbitsai / inference

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
https://inference.readthedocs.io
Apache License 2.0
3.74k stars 317 forks source link

Xinference Dify Integration Issue #860

Open Eng-AliKazemi opened 6 months ago

Eng-AliKazemi commented 6 months ago

dify issue xinference Dify and Xinference are both running and working separately, but when I try to connect to Xinference through Dify, I get this error:

Xinference credentials validate failed: get xinference model extra parameter failed, url: http://127.0.0.1:9997/v1/models/mistral-instruct-v0.2, error: HTTPConnectionPool(host='127.0.0.1', port=9997): Max retries exceeded with url: /v1/models/mistral-instruct-v0.2 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb76ddeeb90>: Failed to establish a new connection: [Errno 111] Connection refused'))

Any help will be appreciated.

aresnow1 commented 6 months ago

what's your version of dify?

gubinjie commented 6 months ago

It seems that the actual IP of the host should be used instead of 127.0.0.1

lpy86786 commented 5 months ago

Same problem. Dify version = 0.5.4, and both Dify and xinference are installed with dockers on an Ubuntu 18.04. The Ubuntu 18.04 is installed in WSL2 of Windows 10.

image I have tried the following IPs but none of them work: http://192.168.103.0:6006(the actual IP) --wait for a long time and get a timeout image http://127.0.0.1:6006 --immediately report the same error image http://localhost:6006 --immediately report the same error image

Besides, I also tried connecting that Dify with LMStudio(Windows). It works fine with IP = http://192.168.103.0:1234/v1, but it does not work with IP = http://localhost:1234/v1 image

lpy86786 commented 5 months ago

I use ifconfig in Ubuntu terminal and tried all the IP addresses listed. Finally I found one capable of usage.