search

hbai98 / SCM

MIT License
28 stars 7 forks source link

Added some features to SCM #3

Open author31 opened 1 year ago

author31 commented 1 year ago

Hi! I really like the your work and I tried that on my RTX 3060 12G, but it kept going NaN so I added lr scheduler and refactor the whole codebase. Here is the link https://github.com/author31/scm_nbdev. Once again, thanks for your work, keep doing.

hbai98 commented 1 year ago

Hi! Thanks for your attention.

Could you please specify where NaN is produced in your code?
Furthermore, can you briefly describe your plan to change the lr scheduler?

author31 commented 1 year ago

In my cases, the NaN problem is because learning rates to large even is 3e-6. So, I added the lr scheduler to do the warmcos scheduling style which like this image