-
### Reminder
- [X] I have read the README and searched the existing issues.
### System Info
absl-py 2.1.0
accelerate 0.32.0
aiofiles …
-
vLLM
We advise you to use vLLM>=0.3.0 to build OpenAI-compatible API service. Start the server with a chat model, e.g. Qwen1.5-7B-Chat:
```
python -m vllm.entrypoints.openai.api_server --model Qw…
-
I have started Qwen API locally using openai_api.py. [https://github.com/QwenLM/Qwen/blob/main/openai_api.py](https://github.com/microsoft/autogen/issues/url)
Tried both examples in:
https://githu…
-
比如我要启一个服务端，我怎么能在接收到请求的时候用上deepspeed的多卡推理呢
-
**例行检查**
[//]: # '方框内填 x 表示打钩'
- [x] 我已确认目前没有类似 issue
- [x] 我已完整查看过项目 README，以及[项目文档](https://doc.fastgpt.in/docs/intro/)
- [x] 我使用了自己的 key，并确认我的 key 是可正常使用的
- [x] 我理解并愿意跟进此 issue，协助测试和提供反馈
…
-
### Self Checks
- [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).
- [X] I have s…
-
### Checklist
- [ ] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.
### Describe the bug
使用lmdeploy lite auto_awq将sft后…
-
测试代码如下
-
### Checked other resources
- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [X] I used the GitHub search to find a sim…
-
**例行检查**
[//]: # '方框内填 x 表示打钩'
- [ ] 我已确认目前没有类似 issue
- [ ] 我已完整查看过项目 README，以及[项目文档](https://doc.fastgpt.in/docs/intro/)
- [ ] 我使用了自己的 key，并确认我的 key 是可正常使用的
- [ ] 我理解并愿意跟进此 issue，协助测试和提供反馈
…