QLoRA微调运行失败[BUG] <title>

QwenLM / Qwen

The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.

Apache License 2.0

13.59k stars 1.11k forks source link

QLoRA微调运行失败[BUG] <title> #1132

Closed xiaohaiqing closed 6 months ago

xiaohaiqing commented 6 months ago

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

[X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

[x] 我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior

No response

期望行为 | Expected Behavior

No response

复现方法 | Steps To Reproduce

from peft import AutoPeftModelForCausalLM from transformers import AutoTokenizer

model = AutoPeftModelForCausalLM.from_pretrained("output_qwen", device_map="auto", trust_remote_code=True).eval() tokenizer = AutoTokenizer.from_pretrained("output_qwen", trust_remote_code=True) response, history = model.chat(tokenizer, "类型#裤风格#英伦风格#简约", history=None) print(response)

运行环境 | Environment

- OS:
- Python:
- Transformers:
- PyTorch:
- CUDA (`python -c 'import torch; print(torch.version.cuda)'`):
accelerate                    0.27.2
aiofiles                      23.2.1
aiohttp                       3.9.3
aiosignal                     1.3.1
altair                        5.2.0
annotated-types               0.6.0
anyio                         4.3.0
attrs                         23.2.0
auto_gptq                     0.7.1
Brotli                        1.0.9
certifi                       2024.2.2
charset-normalizer            2.0.4
click                         8.1.7
coloredlogs                   15.0.1
contourpy                     1.2.0
cycler                        0.12.1
datasets                      2.18.0
deepspeed                     0.14.0
dill                          0.3.8
einops                        0.7.0
fastapi                       0.110.0
ffmpy                         0.3.2
filelock                      3.13.1
fonttools                     4.49.0
frozenlist                    1.4.1
fsspec                        2024.2.0
gekko                         1.0.7
gmpy2                         2.1.2
gradio                        3.41.2
gradio_client                 0.5.0
h11                           0.14.0
hjson                         3.1.0
httpcore                      1.0.4
httpx                         0.27.0
huggingface-hub               0.21.4
humanfriendly                 10.0
idna                          3.4
importlib_resources           6.1.3
Jinja2                        3.1.3
jsonschema                    4.21.1
jsonschema-specifications     2023.12.1
kiwisolver                    1.4.5
latex2mathml                  3.77.0
Markdown                      3.5.2
MarkupSafe                    2.1.3
matplotlib                    3.8.3
mdtex2html                    1.3.0
mkl-fft                       1.3.8
mkl-random                    1.2.4
mkl-service                   2.4.0
mpi4py                        3.1.4
mpmath                        1.3.0
multidict                     6.0.5
multiprocess                  0.70.16
networkx                      3.1
ninja                         1.11.1.1
numpy                         1.26.4
optimum                       1.17.1
orjson                        3.9.15
packaging                     24.0
pandas                        2.2.1
peft                          0.9.0
pillow                        10.2.0
pip                           24.0
protobuf                      4.25.3
psutil                        5.9.8
py-cpuinfo                    9.0.0
pyarrow                       15.0.1
pyarrow-hotfix                0.6
pydantic                      2.6.3
pydantic_core                 2.16.3
pydub                         0.25.1
pynvml                        11.5.0
pyparsing                     3.1.2
PySocks                       1.7.1
python-dateutil               2.9.0.post0
python-multipart              0.0.9
pytz                          2024.1
PyYAML                        6.0.1
referencing                   0.33.0
regex                         2023.12.25
requests                      2.31.0
rouge                         1.0.1
rpds-py                       0.18.0
safetensors                   0.4.2
scipy                         1.12.0
semantic-version              2.10.0
sentencepiece                 0.2.0
setuptools                    68.2.2
six                           1.16.0
sniffio                       1.3.1
starlette                     0.36.3
sympy                         1.12
tiktoken                      0.6.0
tokenizers                    0.13.3
toolz                         0.12.1
torch                         2.2.0
torchaudio                    2.2.0
torchvision                   0.17.0
tqdm                          4.66.2
transformers                  4.32.0
transformers-stream-generator 0.0.4
triton                        2.2.0
typing_extensions             4.9.0
tzdata                        2024.1
urllib3                       2.1.0
uvicorn                       0.28.0
websockets                    11.0.3
wheel                         0.41.2
xxhash                        3.4.1
yarl                          1.9.4

备注 | Anything else?

微信截图_20240311110544

jklj077 commented 6 months ago

First, something seems wrong with the vocab_size (which is the size of the embedding, not the actual vocabulary size) in config.json and the pad_to_multiple_of setting.

首先，在config.json文件中，vocab_size（表示嵌入层大小而非实际词汇表大小）的设置可能存在问题，并且与pad_to_multiple_of设置不匹配。

Second, Target module QuantLinear() is not supported can happen when peft failed to determine the actual AutoGPTQQuantLinear class type, e.g., auto-gptq is not correctly installed or quantization_config is not set accordingly.

其次，Target module QuantLinear() is not supported的错误通常意味着peft无法正确识别到AutoGPTQQuantLinear类的实际类型，例如可能是auto-gptq未被正确安装，或者是quantization_config设置不恰当导致的。

I would suggest provide the content of config.json, lora_config.json, and tokenizer_config.json first, and try downgrading peft<0.8.0 and reinstalling auto-gptq according to their documentation at https://github.com/AutoGPTQ/AutoGPTQ/blob/main/docs/INSTALLATION.md.

针对以上问题，我建议您首先提供config.json、adapter_config.json和tokenizer_config.json这三个配置文件的内容。同时，请尝试将peft降级至0.8.0版本以下并按照https://github.com/AutoGPTQ/AutoGPTQ/blob/main/docs/INSTALLATION.md上的官方文档指导，重新安装auto-gptq。

xiaohaiqing commented 6 months ago

o

你好，我用的是初始环境的配置没有改动过，我把peft降级到0.8.0后，执行测试的时候报了如下错误：

/home/user/anaconda3/envs/qwen-7b-single/lib/python3.11/site-packages/transformers/utils/generic.py:260: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
Try importing flash-attention for faster inference...
Warning: import flash_attn rotary fail, please install FlashAttention rotary to get higher efficiency https://github.com/Dao-AILab/flash-attention/tree/main/csrc/rotary
Warning: import flash_attn rms_norm fail, please install FlashAttention layer_norm to get higher efficiency https://github.com/Dao-AILab/flash-attention/tree/main/csrc/layer_norm
Warning: import flash_attn fail, please install FlashAttention to get higher efficiency https://github.com/Dao-AILab/flash-attention
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:03<00:00,  1.21s/it]
Traceback (most recent call last):
  File "/home/xhq/Qwen/finetune/testQloRA.py", line 4, in <module>
    model = AutoPeftModelForCausalLM.from_pretrained("output_qwen", device_map="auto", trust_remote_code=True).eval()
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/anaconda3/envs/qwen-7b-single/lib/python3.11/site-packages/peft/auto.py", line 124, in from_pretrained
    tokenizer = AutoTokenizer.from_pretrained(pretrained_model_name_or_path)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/anaconda3/envs/qwen-7b-single/lib/python3.11/site-packages/transformers/models/auto/tokenization_auto.py", line 724, in from_pretrained
    raise ValueError(
ValueError: Tokenizer class QWenTokenizer does not exist or is not currently imported.
(qwen-7b-single) [root@adsl-172-10-0-187 finetune]#

peft降级到0.7.0我也试了，错误信息还是跟问题中的一样，我的cuda版本是12.1，auto-gptq版本对应0.7.1我看是没问题的，请问这个可能是什么原因呢？下边是我的config.json、adapter_config.json、tokenizer_config.json adapter_config.json config.json tokenizer_config.json

jklj077 commented 6 months ago

ValueError: Tokenizer class QWenTokenizer does not exist or is not currently imported.

There is a bug in peft 0.8.0 that affects models requiring trust_remote_code=True; please make sure peft<0.8.0.

(In AutoPeftModelForCausalLM.from_pretrained, AutoTokenizer.from_pretrained is called without passing trust_remote_code.)

The same error for peft 0.7.0.

Please double check the version of peft; the error should not occur at the same line. If it does, something is very wrong with your environment.

auto-gptq 0.7.1

The auto-gptq from PyPI is built against PyTorch 2.2.1+cu121. I think you have PyTorch 2.2.0. (Please try pip install auto-gptq==0.7.0) In addition, transformers is too old to support this version of auto-gptq.

The configuration JSON files actually look good to me. To rule out a misconfigured environment, which is my best guess, please try using the provided docker image.

Did you use the same environment to finetune the model?