weight decay确定是0.1吗？

THUDM / AgentTuning

AgentTuning: Enabling Generalized Agent Abilities for LLMs

https://thudm.github.io/AgentTuning/

1.36k stars 95 forks source link

Closed Fu-Dayuan closed 8 months ago

Fu-Dayuan commented 9 months ago

如题，我好像还没有见过这么大的weight decay

Btlmd commented 8 months ago

我检查了训练配置，确实是 0.1

    --weight-decay 1e-1 \