-
### System Info / 系統信息
cuda 12.4，transformers框架
### Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？
- [ ] docker / docker
- [X] pip install / 通过 pip install 安装
- [ ] installation from …
-
### Describe the bug
重启电脑后出现以下问题，重启前多卡是正常运行的
问题：多卡运行模型启动报错，单卡运行正常
### To Reproduce
To help us to reproduce this bug, please provide information below:
1. Your Python version.
3.10
>>> impo…
-
### 是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?
- [X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions
### 该问题是否在FAQ中有解答？ | Is there an existing ans…
-
### 是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?
- [X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions
### 该问题是否在FAQ中有解答？ | Is there an existing ans…
-
Since OpenAI-Compatible API Server now supports image input, can it support these 2 models?
https://github.com/THUDM/CogVLM
https://github.com/QwenLM/Qwen-VL
-
文件地址 https://github.com/QwenLM/Qwen-Agent/blob/main/examples/qwen2vl_function_calling.py
Qwen-Agent 是git clone 本地安装的 时间20240909
使用 https://hub.docker.com/r/qwenllm/qwenvl vllm运行的 7b-int4 model
# …
-
目前有很多自部署API的项目，能否支持一下？
---
**自部署API项目列表**
- **头牌**
- 名称：Kimi Free API
- 开源地址：[https://github.com/LLM-Red-Team/kimi-free-api](https://github.com/LLM-Red-Team/kimi-free-api)
- **大家**
…
-
### Your current environment
```text
tiktoken==0.6.0
transformers==4.38.1
tokenizers==0.15.2
vLLM Version: 0.4.3
fastchat Version: 0.2.36
```
### 🐛 Describe the bug
Currently, I'm using fa…
-
安装教程，使用vllm出错，显卡H100 ， 昨天晚上拉的最新镜像
1、no module 'Qwen2-7B-Instruct',
python -m vllm.entrypoints.openai.api_server --served-model-name Qwen2-VL-7B-Instruct --model model_path
chat_response = …
-
### Bug Description
Integrated with langchain, Qwen-7B-Chat model is deployed under FastChat and vLLM, which OpenAI API is employed. When the number of input tokens is more than 2048, it raise
```…
Hspix updated
9 months ago