-
### System Info / 系統信息
python == 3.9
Name: vllm
Version: 0.5.0.post1
### Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?
- [ ] docker / docker
- [ ] pip install / 通过 pip install 安装…
-
**问题描述 / Problem Description**
模型推理框架为inferencec,使用 summary_file_to_vector_store api 进行知识库内文档总结时,embedding 模型访问正常,但是无法正常访问语言模型
**复现问题的步骤 / Steps to Reproduce**
1. 执行代码:
```python
import json
i…
-
### System Info / 系統信息
* xinference 0.15.2
* torch 2.4.0
torch-complex 0.4.4
torchaudio 2.4.0
torch…
-
### Search before asking
- [X] I had searched in the [issues](https://github.com/eosphoros-ai/DB-GPT/issues?q=is%3Aissue) and found no similar issues.
### Operating system information
Linux
### P…
yuerf updated
2 weeks ago
-
### System Info / 系統信息
python3.10
Ubuntu22.04
### Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?
- [ ] docker / docker
- [X] pip install / 通过 pip install 安装
- [ ] installatio…
-
### System Info / 系統信息
Server error: 503 - [address=0.0.0.0:35434, pid=25922] No available slot found for the model
### Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?
- [ ] docker / d…
-
- [x] This is actually a bug report.
- [x] I am not getting good LLM Results
- [x] I have tried asking for help in the community on discord or discussions and have not received a response.
- [x] I …
-
### Your current environment
```text
The output of `python collect_env.py`
```
### How would you like to use vllm
I use `xinference` to launch model `Qwen1.5-chat`, it use `vllm` in its origin …
-
### System Info / 系統信息
ubuntu22
### Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?
- [X] docker / docker
- [ ] pip install / 通过 pip install 安装
- [ ] installation from source / 从源码安装
…
-
### System Info / 系統信息
xinference v0.15.1(实际上从0.14.0开始一直存在)显卡是A40
### Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?
- [X] docker / docker
- [ ] pip install / 通过 pip install 安装
- [ ] …