-
vLLM was merged into main but there remains runtime depend issues. vLLM is a large pkg we will not force dependency on it. vLLM should runtime import and error if not exists and prompt users to instal…
-
### Checklist
- [ ] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.
- [ ] 3. Please note that if the bug-related issue y…
-
Hello,
First of all thank you for bringing this amazing tool! I was wondering if there is any chance of integrating open-source LMM models like for example https://huggingface.co/Qwen/Qwen2-VL-7B-…
-
### Motivation
Since Llama3.1 is already released. I tested with gptq quant and it doesn't work.
```bash
Traceback (most recent call last):
sglang | File "/usr/lib/python3.10/runpy.py", line…
-
### System Info
x86_64
755G
nvidia T4
ubuntu 22.04
trtllm version : https://github.com/NVIDIA/TensorRT-LLM/archive/9691e12bce7ae1c126c435a049eb516eb119486c.zip
pip install tensorrt-llm==0.11…
-
# System Info
Package Version
------------------------ ----------
accelerate 0.33.0
bitsandbytes 0.43.3
transformers 4.44.0
### Who c…
-
There's been quite a few changes from 0.1. We should document them for people updating their applications.
-
Thank you for your work. However, I've noticed some performance issues that differ significantly when compared to the Llama 3.1 model. Specifically, I've observed the following problems:
# Issue Desc…
-
### Search before asking
- [X] I had searched in the [issues](https://github.com/eosphoros-ai/DB-GPT/issues?q=is%3Aissue) and found no similar issues.
### Operating system information
Linux
### P…
-
请看一下日志我错过了什么？谢谢
根据此部署指南：https://github.com/modelscope/FunASR/blob/main/runtime/docs/SDK_advanced_guide_offline_gpu.md
执行如下命令：
```
docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:f…