-
**Describe the bug**
微调代码如下:
```sh
CUDA_VISIBLE_DEVICES=0 \
swift sft \
--model_type qwen2-7b-instruct-int8 \
--model_id_or_path Qwen2-7B-Instruct-GPTQ-Int8 \
--sft_type lora \
…
-
First, Thanks for this theme, it is great à lot of infos are shown, and beautiful art.
I guess because this grid system view is new you may be working on it, so if any of what i say is on the go, t…
-
### Your current environment
```text
PyTorch version: 2.4.0.dev20240612+rocm6.1
Is debug build: False
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: 6.1.40091-a8dbc0c19
OS: Ubuntu…
-
While running
```
from awq import AutoAWQForCausalLM
from transformers import AutoTokenizer
model_path = 'mistralai/Mixtral-8x7B-Instruct-v0.1'
quant_path = 'mixtral-instruct-awq'
modules_to_n…
-
### Search before asking
- [X] I had searched in the [issues](https://github.com/eosphoros-ai/DB-GPT/issues?q=is%3Aissue) and found no similar issues.
### Operating system information
Linux
### P…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
pfb the code
`
import torch
from transformers import BitsAndBytesConfig
from …
-
### System Info
```Shell
- `Accelerate` version: 0.30.0.dev0
- Platform: Linux-5.15.0-87-generic-x86_64-with-glibc2.35
- `accelerate` bash location: /home/work/.local/anaconda3/envs/multinode-test/…
-
I uploaded my private github repo as data set to private hugging face dataset. Below is the error I receive when I try to train using PEFT method
2024-05-05 18:43:36.206142: E external/local_xla/xl…
-
### Your current environment
pip install vllm
vllm version: 0.5.3post1
### 🐛 Describe the bug
## Trigger
model_params = {
"model": 'google/gemma-2-9b-it',
"quantization": "fp8",
…
-
### System Info
CPU: x86_64
GPU: 4060 RTX
CUDA Version: 12.2.0
TensorRT-LLM version: TensorRT-LLM 0.8.0
On branch rel
Driver Version: 546.12
Python: 3.10
OS : Windows 11
##…