-
### Your current environment
[Bug]: 4208 CPU vllm 0.6.0 启动qwen-vl-7b ,报下面图片中的异常，模型开始可以正常输出，调用多次后，无返回结果
### Model Input Dumps
[Bug]: 4208 CPU vllm
, which supports `functions` in a manner similar to how GPT operates through the OpenAI API. I've successfully dep…
-
我在部署qwen1.5-7B-Chat的时候遇到调用API时最后有10个字符缺失的问题，长度正好是结束token。
```
nohup python -m vllm.entrypoints.openai.api_server \
--model /Qwen/Qwen1.5-7B-Chat
--host 0.0.0.0 \
--port 80 \
--trust-remote-c…
-
### 先决条件
- [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。
- [X] 错误在 [最新版本](https://github.com/open-com…
-
### Your current environment
The output of `python collect_env.py`
```text
Collecting environment information...
2024-11-04 10:14:53.454810: E external/local_xla/xla/stream_executor/cuda/cud…
-
```log
~/repo/FastChat$ python -m fastchat.serve.model_worker --model-path ~/repo/models/Qwen-14B-Chat-Int4 --gptq-wbits 4 --gptq-groupsize 128 --model-names gpt-3.5-turbo
2023-09-28 14:36:05 | INFO…
-
比如我想用 [kimi-free-api](https://github.com/LLM-Red-Team/kimi-free-api) 和 [qwen-free-api](https://github.com/LLM-Red-Team/qwen-free-api) ，目前需要分别启用两个 docker 服务，占用空间较大，如果后面还有类似服务，操作也略微繁琐。
希望能够整合成一个服务，复用…
-
### 先决条件
- [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。
- [X] 错误在 [最新版本](https://github.com/open-com…
-
### Checklist
- [X] 1. I have searched related issues but cannot get the expected help.
- [X] 2. The bug has not been fixed in the latest version.
### Describe the bug
When quantizing Qwen2-…
-
