Finetune Qwen-Audio-Chat时出错

[lucky-dreamer-nn]

运行命令：bash scripts/qwen_audio_chat/lora/sft.sh [INFO:swift] training_args: Seq2SeqTrainingArguments打印后开启训练时报错： File "llm_sft.py", line 7, in output = sft_main() File "/datalocal/swift-main/swift/utils/run_utils.py", line 27, in x_main return llm_x(args, kwargs) File "/datalocal/swift-main/swift/llm/sft.py", line 280, in llm_sft trainer.train(training_args.resume_from_checkpoint) File "/root/anaconda3/envs/modelscope/lib/python3.8/site-packages/transformers/trainer.py", line 1553, in train return inner_training_loop( File "/root/anaconda3/envs/modelscope/lib/python3.8/site-packages/transformers/trainer.py", line 1835, in _inner_training_loop tr_loss_step = self.training_step(model, inputs) File "/datalocal/swift-main/swift/trainers/trainers.py", line 47, in training_step training_output = super().training_step(args, kwargs) File "/root/anaconda3/envs/modelscope/lib/python3.8/site-packages/transformers/trainer.py", line 2679, in training_step loss = self.compute_loss(model, inputs) File "/datalocal/swift-main/swift/trainers/trainers.py", line 183, in compute_loss loss, outputs = super().compute_loss(model, inputs, True) File "/root/anaconda3/envs/modelscope/lib/python3.8/site-packages/transformers/trainer.py", line 2704, in compute_loss outputs = model(inputs) File "/root/anaconda3/envs/modelscope/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/root/anaconda3/envs/modelscope/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, kwargs) File "/root/anaconda3/envs/modelscope/lib/python3.8/site-packages/accelerate/utils/operations.py", line 680, in forward return model_forward(*args, *kwargs) File "/root/anaconda3/envs/modelscope/lib/python3.8/site-packages/accelerate/utils/operations.py", line 668, in call return convert_to_fp32(self.model_forward(args, kwargs)) File "/root/anaconda3/envs/modelscope/lib/python3.8/site-packages/torch/amp/autocast_mode.py", line 16, in decorate_autocast return func(*args, kwargs) File "/datalocal/swift-main/swift/tuners/base.py", line 107, in forward return self.base_model(*args, *kwargs) File "/root/anaconda3/envs/modelscope/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/root/anaconda3/envs/modelscope/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, kwargs) File "/root/.cache/huggingface/modules/transformers_modules/Qwen-Audio-Chat/modeling_qwen.py", line 1107, in forward transformer_outputs = self.transformer( File "/root/anaconda3/envs/modelscope/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/root/anaconda3/envs/modelscope/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(args, kwargs) File "/root/.cache/huggingface/modules/transformers_modules/Qwen-Audio-Chat/modeling_qwen.py", line 880, in forward hidden_states[i][a : b+1] = audios[idx] RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation.

[Jintao-Huang]

你重新拉一下最新的代码哈, 这个bug已经修复了😊. 你可以尝试在sh的最后加上--lazy_tokenize true来减少预处理的等待

[lucky-dreamer-nn]

多谢，拉取最新的代码可以跑起来了