search

THUDM / ChatGLM2-6B

ChatGLM2-6B: An Open Bilingual Chat LLM | 开源双语对话语言模型
Other
15.65k stars 1.85k forks source link

[Help]有没有人尝试过把优化器改成SGD来减少显存占用的 #633

Open 31-ryougishiki opened 8 months ago

31-ryougishiki commented 8 months ago

Is there an existing issue for this?

Current Behavior

我看代码里使用的优化器好像是transformers自带的adamw，是不是可以换成SGD减少显存。

Expected Behavior

No response

Steps To Reproduce

无

Environment

无

Anything else?

No response

hhy150 commented 6 months ago

你好，请问在哪里看到优化器的设置呢？