Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?
[ ] docker / docker
[X] pip install / 通过 pip install 安装
[ ] installation from source / 从源码安装
Version info / 版本信息
最新版本
The command used to start Xinference / 用以启动 xinference 的命令
xinference-local --host 0.0.0.0 --port 9997
Reproduction / 复现过程
(xinference) test@Super-Server:~$ XINFERENCE_MODEL_SRC=modelscope xinference-local --host 0.0.0.0 --port 9997
fatal: bad revision 'HEAD'
2024-09-22 15:41:12,380 xinference.core.supervisor 1738769 INFO Xinference supervisor 0.0.0.0:41562 started
2024-09-22 15:41:19,942 xinference.core.worker 1738769 INFO Starting metrics export server at 0.0.0.0:None
2024-09-22 15:41:19,947 xinference.core.worker 1738769 INFO Checking metrics export server...
2024-09-22 15:41:22,892 xinference.core.worker 1738769 INFO Metrics server is started at: http://0.0.0.0:42699
2024-09-22 15:41:22,892 xinference.core.worker 1738769 INFO Purge cache directory: /home/test/.xinference/cache
2024-09-22 15:41:22,894 xinference.core.worker 1738769 INFO Connected to supervisor as a fresh worker
2024-09-22 15:41:22,902 xinference.core.worker 1738769 INFO Xinference worker 0.0.0.0:41562 started
2024-09-22 15:41:24,284 xinference.api.restful_api 1738616 INFO Starting Xinference at endpoint: http://0.0.0.0:9997
2024-09-22 15:41:24,384 uvicorn.error 1738616 INFO Uvicorn running on http://0.0.0.0:9997 (Press CTRL+C to quit)
System Info / 系統信息
cuda:12.2 llama-cpp-python: llama_cpp_python-0.2.88-cp311-cp311-linux_x86_64.whl python: 3.11 Ubuntu:22.04.4
Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?
Version info / 版本信息
最新版本
The command used to start Xinference / 用以启动 xinference 的命令
xinference-local --host 0.0.0.0 --port 9997
Reproduction / 复现过程
(xinference) test@Super-Server:~$ XINFERENCE_MODEL_SRC=modelscope xinference-local --host 0.0.0.0 --port 9997 fatal: bad revision 'HEAD' 2024-09-22 15:41:12,380 xinference.core.supervisor 1738769 INFO Xinference supervisor 0.0.0.0:41562 started 2024-09-22 15:41:19,942 xinference.core.worker 1738769 INFO Starting metrics export server at 0.0.0.0:None 2024-09-22 15:41:19,947 xinference.core.worker 1738769 INFO Checking metrics export server... 2024-09-22 15:41:22,892 xinference.core.worker 1738769 INFO Metrics server is started at: http://0.0.0.0:42699 2024-09-22 15:41:22,892 xinference.core.worker 1738769 INFO Purge cache directory: /home/test/.xinference/cache 2024-09-22 15:41:22,894 xinference.core.worker 1738769 INFO Connected to supervisor as a fresh worker 2024-09-22 15:41:22,902 xinference.core.worker 1738769 INFO Xinference worker 0.0.0.0:41562 started 2024-09-22 15:41:24,284 xinference.api.restful_api 1738616 INFO Starting Xinference at endpoint: http://0.0.0.0:9997 2024-09-22 15:41:24,384 uvicorn.error 1738616 INFO Uvicorn running on http://0.0.0.0:9997 (Press CTRL+C to quit)
Expected behavior / 期待表现
浏览器打开//host-ip:9997/应该可以看到xinference的界面,但是我现在看不到