search

xorbitsai / inference

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
https://inference.readthedocs.io
Apache License 2.0
4.97k stars 395 forks source link

Can xinference launch surpose the image model such as stable-diffusion-2-1 which is local pattern? #1196

Closed Songjiadong closed 1 month ago

Songjiadong commented 6 months ago

can xinference launch surpose the image model which is local pattern？ this is my shell 

 xinference launch -u miic_graph -n  /usr/local/models/stabilityai/stable-diffusion-2-1 -t image -e "http://127.0.0.1:8888"

result:

RuntimeError: Failed to launch model, detail: [address=0.0.0.0:32036, pid=177526] Image model /usr/local/models/stabilityai/stable-diffusion-2-1 not found, availablemodel list: dict_keys(['sd-turbo', 'sdxl-turbo', 'stable-diffusion-v1.5', 'stable-diffusion-xl-base-1.0'])
Songjiadong commented 6 months ago

if it is supporsed,can we give me a simple demo?

qinxuye commented 6 months ago

This is a feature we want to add, that is like --model-path xxx, which just uses the default model but with a local path.

This is not implemented yet.

Songjiadong commented 6 months ago

@qinxuye Approximately when will this feature be released？

qinxuye commented 6 months ago

This is only work for xinference-local, we plan to add this in 2 weeks approximately, but PR welcomed.

qinxuye commented 6 months ago

After discussion internally, we found this feature is underestimated, we may need some refactor, so 2 weeks is not enough, welcome to give advice anyway.

github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 7 days with no activity.

github-actions[bot] commented 1 month ago

This issue was closed because it has been inactive for 5 days since being marked as stale.