Closed acbp1379 closed 4 months ago
model = AutoModelForCausalLM.from_pretrained(model_args.model_name_or_path, config=config, **init_kwargs)
File "/root/anaconda3/envs/llama_factory/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 563, in from_pretrained return model_class.from_pretrained( File "/root/anaconda3/envs/llama_factory/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3044, in from_pretrained hf_quantizer = AutoHfQuantizer.from_config(config.quantization_config, pre_quantized=pre_quantized) File "/root/anaconda3/envs/llama_factory/lib/python3.10/site-packages/transformers/quantizers/auto.py", line 124, in from_config return target_cls(quantization_config, **kwargs) File "/root/anaconda3/envs/llama_factory/lib/python3.10/site-packages/transformers/quantizers/quantizer_gptq.py", line 47, in init from optimum.gptq import GPTQQuantizer ModuleNotFoundError: No module named 'optimum'
optimum=1.16.2 optimum-graphcore=0.3.1
可能环境不一致
可能环境不一致
我是在单机训练的,您指的是哪方面不一致呢?
有办法合并微调后lora参数和原始量化模型参数么?
有办法合并微调后lora参数和原始量化模型参数么?
有办法合并微调后lora参数和原始量化模型参数么?
你的意思是不能lora微调量化版的模型吗?我这是第一次尝试微调量化版模型
咨询一下Qwen1.5-32B-Chat-GPTQ-Int4微调的参数能不能合并,我上次问过qwen的,说合并不了
咨询一下Qwen1.5-32B-Chat-GPTQ-Int4微调的参数能不能合并,我上次问过qwen的,说合并不了
@hiyouga 没法合并,是不是这样就没法量化模型的lora微调了
Reminder
Reproduction
ModuleNotFoundError: No module named 'optimum' 但是我的python环境里面是有这个包的。 deepspeed --num_gpus 3 src/train_bash.py \ --deepspeed examples/deepspeed/ds_z2_config.json \ --stage sft \ --do_train \ --model_name_or_path /home/models/Qwen/Qwen1.5-32B-Chat-GPTQ-Int4/ \ --dataset alpaca_gpt4_en \ --dataset_dir data \ --template default \ --finetuning_type lora \ --lora_target q_proj,v_proj \ --output_dir saves/qwen1.5-32B/sft/ \ --overwrite_cache \ --overwrite_output_dir \ --cutoff_len 1024 \ --per_device_train_batch_size 1 \ --per_device_eval_batch_size 1 \ --gradient_accumulation_steps 8 \ --lr_scheduler_type cosine \ --logging_steps 10 \ --save_steps 100 \ --eval_steps 100 \ --evaluation_strategy steps \ --load_best_model_at_end \ --learning_rate 5e-5 \ --num_train_epochs 3.0 \ --max_samples 3000 \ --val_size 0.1 \ --plot_loss \ --fp16
Expected behavior
No response
System Info
No response
Others
No response