search

lessw2020 / Ranger21

Ranger deep learning optimizer rewrite to use newest components
Apache License 2.0
323 stars 46 forks source link

error when training with batch_size = 1 #20

Open neuronflow opened 3 years ago

neuronflow commented 3 years ago

error when training with batch_size = 1

  File "/home/florian/miniconda3/envs/msblob/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context
    return func(*args, **kwargs)
  File "/home/florian/miniconda3/envs/msblob/lib/python3.8/site-packages/ranger21/ranger21.py", line 680, in step
    raise RuntimeError("hit nan for variance_normalized")
RuntimeError: hit nan for variance_normalized