Closed gk-cv closed 1 day ago
请问配置问题吗
I don't think it's a configuration issue per se. It's just that script is not expected to be used for finetuning in that scale. You'd be better use dedicated finetuning frameworks as mentioned in the README. You may want to also try pretraining framework like mcore for that kind of finetuning.
This issue has been automatically marked as inactive due to lack of recent activity. Should you believe it remains unresolved and warrants attention, kindly leave a comment on this thread.
参考这里https://qwen.readthedocs.io/zh-cn/latest/training/SFT/example.html脚本
使用 24张A100,对7B sft,model_max_length超过2w时,OOM