Closed ntz2000 closed 3 days ago
lora微调Qwen-14b-chat后和原始模型merge，在导入merge后的模型时报错： ModuleNotFoundError: No module named 'transformers_modules.2' 我用的是transformers 4.41.2 + python3.10 + torch 2.3.0
model_name_or_path: /LLM/Qwen/models/Qwen-14B-Chat
stage: sft do_train: true finetuning_type: lora lora_target: all deepspeed: examples/deepspeed/ds_z3_config.json
dataset: sft_data_test template: qwen cutoff_len: 1024 max_samples: 1000 overwrite_cache: true preprocessing_num_workers: 16
output_dir: /LLM/results/2.0version-Qwen-14b-lora-adapt-continue-pinpai-3eps logging_steps: 10 save_steps: 500 plot_loss: true overwrite_output_dir: true
per_device_train_batch_size: 32 gradient_accumulation_steps: 8 learning_rate: 1.0e-4 num_train_epochs: 3.0 lr_scheduler_type: cosine warmup_ratio: 0.1 fp16: true ddp_timeout: 180000000
val_size: 0.1 per_device_eval_batch_size: 1 eval_strategy: steps eval_steps: 500
No response
路径不要加 .
hiyouga
commented
3 days ago hiyouga
commented
3 days ago
Reminder
System Info
lora微调Qwen-14b-chat后和原始模型merge，在导入merge后的模型时报错： ModuleNotFoundError: No module named 'transformers_modules.2' 我用的是transformers 4.41.2 + python3.10 + torch 2.3.0
Reproduction
model
model_name_or_path: /LLM/Qwen/models/Qwen-14B-Chat
method
stage: sft do_train: true finetuning_type: lora lora_target: all deepspeed: examples/deepspeed/ds_z3_config.json
dataset
dataset: sft_data_test template: qwen cutoff_len: 1024 max_samples: 1000 overwrite_cache: true preprocessing_num_workers: 16
output
output_dir: /LLM/results/2.0version-Qwen-14b-lora-adapt-continue-pinpai-3eps logging_steps: 10 save_steps: 500 plot_loss: true overwrite_output_dir: true
train
per_device_train_batch_size: 32 gradient_accumulation_steps: 8 learning_rate: 1.0e-4 num_train_epochs: 3.0 lr_scheduler_type: cosine warmup_ratio: 0.1 fp16: true ddp_timeout: 180000000
eval
val_size: 0.1 per_device_eval_batch_size: 1 eval_strategy: steps eval_steps: 500
Expected behavior
No response
Others
No response