使用cogvlm2在rlaif-v数据集上做DPO训练报错

[kaka-Cao]

在训练前，使用modelscope download --dataset swift/RLAIF-V-Dataset 命令下载了数据集，保存路径为/home/xxx/.cache/modelscope/datasets/swift/RLAIF-V-Dataset，刚刚才拉取了最新的swift main分支。运行脚本如下： CUDA_VISIBLE_DEVICES=0 \ swift rlhf \ --rlhf_type dpo \ --model_type cogvlm2-19b-chat \ --model_id_or_path /home/data3/zys/MLLM/pretrained_models/cogvlm2-llama3-chinese-chat-19B/ \ --beta 0.1 \ --sft_beta 0.1 \ --sft_type lora \ --dataset rlaif-v#1000 \ --num_train_epochs 2 \ --lora_target_modules DEFAULT \ --gradient_checkpointing false \ --batch_size 1 \ --learning_rate 5e-5 \ --gradient_accumulation_steps 16 \ --warmup_ratio 0.03 \ --save_total_limit 2 报错如下： Original Traceback (most recent call last): File "/home/zhangyusi/.conda/envs/MLLM/lib/python3.12/site-packages/torch/utils/data/_utils/worker.py", line 309, in _worker_loop
data = fetcher.fetch(index) # type: ignore[possibly-undefined]
File "/home/zhangyusi/.conda/envs/MLLM/lib/python3.12/site-packages/torch/utils/data/_utils/fetch.py", line 55, in fetch return self.collate_fn(data)
File "/home/zhangyusi/swift/swift/trainers/utils.py", line 424, in new_call
padded_batch[k] = [torch.Tensor(ex[k]) for ex in features]
ValueError: only one element tensors can be converted to Python scalars 版本： torch 2.4.0 CUDA 12.1 transformers 4.41.0

[SparrowZheyuan18]

I encounter the same issue with my own dataset

[Jintao-Huang]

https://github.com/modelscope/ms-swift/pull/1975