`TypeError: RAdam() got an unexpected keyword argument 'sqrmom'`

[gsganden]

Describe the bug The fine tuning steps raise the error TypeError: RAdam() got an unexpected keyword argument 'sqrmom' in at least the SimCLR and SwAV tutorials (e.g. finetune(size, epochs=5, arch='xresnet34', encoder_path=f'models/swav_iwang_sz{size}_epc100_encoder.pth') in https://keremturgutlu.github.io/self_supervised/training_swav_iwang.html).

To Reproduce Steps to reproduce the behavior:

1. Open https://colab.research.google.com/github/KeremTurgutlu/self_supervised/blob/master/examples/training_swav_iwang.ipynb
2. Go to Runtime > Change runtime type in the menu and set Hardware accelerator to GPU.
3. Create a new code cell at the top with contents !pip install self-supervised.
4. Comment out from fastai.callback.wandb import WandbCallback and import wandb.
5. Change bs, resize, size = 96, 256, 224 to bs, resize, size = 48, 256, 224 to avoid a CUDA out of memory error.
6. Optional: change lr,wd,epochs=1e-2,1e-2,100 to lr,wd,epochs=1e-2,1e-2,1 and finetune(size, epochs=5, arch='xresnet34', encoder_path=f'models/swav_iwang_sz{size}_epc100_encoder.pth' to finetune(size, epochs=5, arch='xresnet34', encoder_path=f'models/swav_iwang_sz{size}_epc1_encoder.pth' to get to the error much faster.
7. Run all cells.

Expected behavior I expect the model to train without errors.

Error with full stack trace

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-27-30032d6c1d48> in <module>()
      1 acc = []
      2 runs = 5
----> 3 for i in range(runs): acc += [finetune(size, epochs=5, arch='xresnet34', encoder_path=f'models/swav_iwang_sz{size}_epc1_encoder.pth')]

4 frames
/usr/local/lib/python3.7/dist-packages/fastai/optimizer.py in ranger(p, lr, mom, wd, eps, **kwargs)
    319 def ranger(p, lr, mom=0.95, wd=0.01, eps=1e-6, **kwargs):
    320     "Convenience method for `Lookahead` with `RAdam`"
--> 321     return Lookahead(RAdam(p, lr=lr, mom=mom, wd=wd, eps=eps, **kwargs))
    322 
    323 # Cell

TypeError: RAdam() got an unexpected keyword argument 'sqrmom'

Additional context The notebook runs fine if you remove sqrmom from optdict = dict(sqrmom=0.99,mom=0.95,beta=0.,eps=1e-4), but I suspect the preferred solution involves calling a different function that accepts sqrmom.

I also get this error when I run the notebook locally after pip install self-supervised in a fresh pyenv-virtualenv environment.

My library versions look right to me:

[gsganden]

It looks like we just need to change sqrmom to sqr_mom. I'll submit a PR.

[gsganden]

Tangential suggestion: put from fastai.callback.wandb import WandbCallback and import wandb under if WANDB so that folks who haven't installed wandb don't have to change anything.

[KeremTurgutlu]

Thanks for reminding me of this, I will be making these changes on the current branch I am working on.