search

juntang-zhuang / Adabelief-Optimizer

Repository for NeurIPS 2020 Spotlight "AdaBelief Optimizer: Adapting stepsizes by the belief in observed gradients"
BSD 2-Clause "Simplified" License
1.05k stars 109 forks source link

keyerror exp_avg_var #7

Closed mcmingchang closed 4 years ago

mcmingchang commented 4 years ago

pytorch 1.6
keyerror exp_avg_var

juntang-zhuang commented 4 years ago

thanks for feedback, could you provide more details, such as which version? If possible please copy the error message here. will look into it later.

juntang-zhuang commented 4 years ago

I tested with PyTorch 16 and adabelief-pytorch 0.0.5, seems works fine. Did you use some older version?

mcmingchang commented 4 years ago

train centernet and use resume

---Original--- From: "Juntang Zhuang"<notifications@github.com> Date: Thu, Oct 22, 2020 00:27 AM To: "juntang-zhuang/Adabelief-Optimizer"<Adabelief-Optimizer@noreply.github.com>; Cc: "mcmingchang"<355152992@qq.com>;"Author"<author@noreply.github.com>; Subject: Re: [juntang-zhuang/Adabelief-Optimizer] keyerror exp_avg_var (#7)

I tested with PyTorch 16 and adabelief-pytorch 0.0.5, seems works fine. Did you use some older version?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

mcmingchang commented 4 years ago

在训练centernet的时候，之前是用adam，更新优化器后再resume出现上述问题

juntang-zhuang commented 4 years ago

You can not simply resume adabelief from an adam, though you can resume model weights. Model keys in the states of optimizers are different. PS, epsilon for adabelief needs to bet set roughly as the square of eps in Adam if you want them to behave similar.