-
### System Info
peft: 0.10.1.dev0
accelerate: 0.30.0
bitsandbytes: 0.43.1
transformers: 4.39.3
GPU: A6000 * 2 ( 96GB )
nvidia-driver version: 535.171.04
cuda: 11.8
### Who can help?
_No…
-
### Your current environment
```text
PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: N/A
OS: Amazon Linux 2 (x86_64)
GCC version: (GCC) 7.…
-
### System Info
nvidia/cuda:12.1.0-devel-ubuntu22.04
![image](https://github.com/NVIDIA/TensorRT-LLM/assets/77143808/263e3278-cf94-4266-99f4-9872eb445e21)
### Who can help?
_No response_
### In…
-
**Describe the bug**
参考代码,量化 qwen72Bchat模型,在量化过程中 出现问题
2024-04-11 06:23:14 INFO [auto_gptq.modeling._base] Quantizing mlp.c_proj in layer 8/80...
Traceback (most recent call last):
File "/data/q…
-
https://hf-mirror.com/SakuraLLM/Sakura-13B-LNovel-v0.9/tree/main
前几天下了0.9,把0.9模型的文件夹放在`Sakura-13B-Galgame\models`路径下,conda单独创建了一个环境,执行以下命令:
```CMD
pip3 install torch torchvision torchaudio --inde…
-
### Checked other resources
- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [x] I used the GitHub search to find a sim…
-
Hi, congratulations for the work done. I have a problem with the script. After following the instructions for installing the package, even if I execute the command "sh aircloud.sh websocatdebug" fro…
-
### System Info / 系統信息
cuda12.1
torch 2.3.0
transformers 4.42.4
vllm 0.5.1
xinference 0.13.1…
-
vllm在我的机器上多卡似乎不太可行,环境要求有点苛刻,希望可以给出不基于vllm的构建api方式
-
I am experiencing the same issues when switching between multiple adapters despite following the [documentation](https://huggingface.co/docs/transformers/main/en/peft#enable-and-disable-adapters) and …