search

erfanzar / EasyDeL

Accelerate, Optimize performance with streamlined training and serving options with JAX.
https://easydel.readthedocs.io/en/latest/
Apache License 2.0
192 stars 23 forks source link

Resume from checkpoint #46

Closed JinSeoungwoo closed 9 months ago

JinSeoungwoo commented 10 months ago

When I load model from a checkpoint and try to continue training at the previous learning rate, the loss increases sharply after a small step, so I have to warm up. Is this normal?

erfanzar commented 10 months ago

hi in case that you have changed the training configs such as optimizer , learning rate or dataset yes this is normal