SimPO微调报错

[zhangfan-algo]

Describe the bug

Your hardware and system info torchrun --nproc_per_node ${num_gpu_per_node} --master_port $MASTER_PORT --master_addr $MASTER_ADDR --node_rank $RANK --nnodes $WORLD_SIZE examples/pytorch/llm/llm_rlhf.py \ --model_cache_dir /mnt/cluster/models/Qwen/Qwen1.5-1.8B-Chat \ --model_type qwen1half-1_8b-chat \ --rlhf_type simpo \ --sft_type full \ --beta 2.0 \ --simpo_gamma 1.0 \ --tuner_backend swift \ --template_type AUTO \ --ddp_backend nccl \ --custom_train_dataset_path /mnt/cluster/test.jsonl \ --output_dir /mnt/cluster/swift_0522/output/\ --preprocess_num_proc 60 \ --dataloader_num_workers 60 \ --train_dataset_sample -1 \ --evaluation_strategy steps \ --eval_steps 50 \ --eval_batch_size 1 \ --dataset_test_ratio 0.01 \ --max_length 19500 \ --max_new_tokens 1500 \ --lr_scheduler_type cosine \ --num_train_epochs 5 \ --save_total_limit 5 \ --save_strategy epoch \ --logging_steps 10 \ --batch_size 1 \ --check_dataset_strategy warning \ --gradient_checkpointing true \ --gradient_accumulation_steps 8 \ --weight_decay 0.01 \ --learning_rate 1e-5 \ --max_grad_norm 0.5 \ --warmup_ratio 0.03 \ --use_flash_attn true \ --push_to_hub false \ --lazy_tokenize true \ --deepspeed_config_path /mnt/cluster/zhangfan/study_info/swift_0619/swift/llm/ds_config/zero3_offload.json \ --save_only_model true \ --save_on_each_node false \ --neftune_noise_alpha 5 \ --dtype AUTO

[zhangfan-algo]

数据集格式为:{'system':system,"query":q,'response':l,"rejected_response":r}类似这样的

[zhangfan-algo]

2024-06-21 18:35:18 Traceback (most recent call last): 2024-06-21 18:35:18 File "/mnt/cluster/zhangfan/study_info/swift_0621/examples/pytorch/llm/llm_rlhf.py", line 18, in 2024-06-21 18:35:18 output = rlhf_main() 2024-06-21 18:35:18 File "/root/anaconda3/envs/swift/lib/python3.10/site-packages/swift/utils/run_utils.py", line 27, in x_main 2024-06-21 18:35:18 result = llm_x(args, **kwargs) 2024-06-21 18:35:18 File "/root/anaconda3/envs/swift/lib/python3.10/site-packages/swift/llm/rlhf.py", line 166, in llm_rlhf 2024-06-21 18:35:18 train_dataset, val_dataset = get_dataset( 2024-06-21 18:35:18 File "/root/anaconda3/envs/swift/lib/python3.10/site-packages/swift/llm/utils/dataset.py", line 2281, in get_dataset 2024-06-21 18:35:18 train_d = _reduce_dataset(train_d) 2024-06-21 18:35:18 File "/root/anaconda3/envs/swift/lib/python3.10/site-packages/swift/llm/utils/dataset.py", line 2266, in _reduce_dataset 2024-06-21 18:35:18 features['_system'] = Value(dtype='string') 2024-06-21 18:35:18 TypeError: 'NoneType' object does not support item assignment

[hjh0119]

这个问题已经修复了 更新下