Question about --dpo_ftx Parameter Setting

hiyouga / LLaMA-Factory

Unified Efficient Fine-Tuning of 100+ LLMs (ACL 2024)

https://arxiv.org/abs/2403.13372

Apache License 2.0

32.62k stars 3.99k forks source link

Question about --dpo_ftx Parameter Setting #3781

Closed zandfj closed 5 months ago

zandfj commented 5 months ago

Reminder

[X] I have read the README and searched the existing issues.

Reproduction

The --dpo_ftx parameter introduces sftloss. Based on experience, what would be a good setting for --dpo_ftx? Should it generally scale sftloss to a level similar in magnitude to dpoloss, or is there a specific fixed --dpo_ftx setting that consistently yields good results?

Expected behavior

No response

System Info

No response

Others

No response

hiyouga commented 5 months ago

We do not have optimal values, try dpo_ftx from 0.1 to 10