Closed YuhuiMa closed 4 years ago
Use normalize=True
Hello,
As @chledowski suggested you can test normalize=True at least for the initial iterations of the optimization process. I have pushed some changes to the code which should help significantly in the stability of the optimization in the initial iterations.
I would ask you to test those changes and re-open an issue if you are still facing problems with the optimization. Thank you for your feedback.
How to deal with the problem of nan loss during training?