search

QwenLM / Qwen2.5

Qwen2.5 is the large language model series developed by Qwen team, Alibaba Cloud.
9.44k stars 582 forks source link

使用Qwen2的sft脚本微调Qwen2-1.5B-Instruct-GPTQ-In4模型失败报错：AttributeError: 'BitsAndBytesConfig' object has no attribute 'get_loading_attributes' #795

Closed adi-hld closed 2 months ago

adi-hld commented 3 months ago

命令行： /mnt/workspace/Qwen2/examples/sft# bash finetune.sh -m /mnt/workspace/.cache/modelscope/hub/qwen/Qwen2-1___5B-Instruct-GPTQ-Int4 -d ../../../data.jsonl --deepspeed ds_config_zero2.json --use_lora True --q_lora True 使用镜像：modelscope环境 image 已经按照https://qwen.readthedocs.io/zh-cn/latest/training/SFT/example.html 安装了微调需要的模块 pip install peft deepspeed optimum accelerate 报错 _AttributeError: 'BitsAndBytesConfig' object has no attribute 'get_loadingattributes' image

jklj077 commented 3 months ago

hi, that script runs QLoRA with bitsandbytes for quantization and it is not possible to quantize a quantized model. Please change the model to Qwen2-1.5B-Instruct.

github-actions[bot] commented 2 months ago

This issue has been automatically marked as inactive due to lack of recent activity. Should you believe it remains unresolved and warrants attention, kindly leave a comment on this thread.