[Feature] dpo场景 为什么 onlinedpo效果不如dpo，rlhf会有下降？

[yiyepiaoling0715]

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like A clear and concise description of what you want to happen.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

[adoda]

开源HH数据集中没有coding相关数据，这块效果会比较随机。这里关注avg指标即可。生产训练中，可以按需准备训练数据。