Closed onlyfish79 closed 2 days ago
llamafactory
torchrun --nproc_per_node=8 --master_port=20001 src/train.py \ --stage dpo \ --pref_loss simpo \ --pref_beta 2.0 \ --simpo_gamma 1.0 \ --do_train \ --model_name_or_path /data/basemodel/Yi-34B \ --dataset novel \ --val_size 0.05 \ --dataset_dir /data/train_data \ --overwrite_cache \ --template yi \ --finetuning_type full \ --output_dir /data/dpo_model/yi34b_simpo_full \ --overwrite_output_dir \ --preprocessing_num_workers 128 \ --per_device_train_batch_size 4 \ --per_device_eval_batch_size 8 \ --gradient_accumulation_steps 4 \ --eval_strategy steps \ --eval_steps 10 \ --save_strategy steps \ --save_total_limit 1 \ --lr_scheduler_type cosine \ --logging_steps 1 \ --save_steps 230 \ --learning_rate 1e-5 \ --num_train_epochs 1.0 \ --max_grad_norm 0.5 \ --weight_decay 0 \ --warmup_ratio 0.03 \ --cutoff_len 2048 \ --plot_loss \ --bf16 \ --flash_attn fa2 \ --gradient_checkpointing \ --ddp_timeout 180000000 \ --deepspeed deepspeed/ds_zero3_offload_optimizer_param.json
ds_zero3_offload_optimizer_param.json
{ "bf16": { "enabled": "auto" }, "optimizer": { "type": "AdamW", "params": { "lr": "auto", "betas": "auto", "eps": "auto", "weight_decay": "auto" } }, "scheduler": { "type": "WarmupDecayLR", "params": { "total_num_steps": "auto", "warmup_min_lr": "auto", "warmup_max_lr": "auto", "warmup_num_steps": "auto" } }, "zero_optimization": { "stage": 3, "offload_optimizer": { "device": "cpu", "pin_memory": true }, "offload_param": { "device": "cpu", "pin_memory": true }, "overlap_comm": true, "contiguous_gradients": true, "sub_group_size": 1e9, "reduce_bucket_size": "auto", "stage3_prefetch_bucket_size": "auto", "stage3_param_persistence_threshold": "auto", "stage3_max_live_parameters": 1e9, "stage3_max_reuse_distance": 1e9, "stage3_gather_16bit_weights_on_model_save": true }, "gradient_accumulation_steps": "auto", "gradient_clipping": "auto", "steps_per_print": 50, "train_batch_size": "auto", "train_micro_batch_size_per_gpu": "auto", "wall_clock_breakdown": false }
数据集字段:instruction, chosen, rejected
任务:小说创作,输入:提供一个小说片段,输出:2/3个选项分支
问题描述:
问题: 请问是因为Simpo不适合小说创作任务吗?
期望返回的结果形如下:
No response
Reminder
System Info
llamafactory
version: 0.8.3.dev0Reproduction
ds_zero3_offload_optimizer_param.json
数据集字段:instruction, chosen, rejected
任务:小说创作,输入:提供一个小说片段,输出:2/3个选项分支
问题描述:
问题: 请问是因为Simpo不适合小说创作任务吗?
Expected behavior
期望返回的结果形如下:![image](https://github.com/hiyouga/LLaMA-Factory/assets/18341845/0b15de35-cb8e-4109-b7d3-26e6f9d9aa22)
Others
No response