Pytorch 1.6 warning - Githubissues

Luolc / AdaBound

An optimizer that trains as fast as Adam and as good as SGD.

https://www.luolc.com/publications/adabound/

Apache License 2.0

2.9k stars 330 forks source link

Pytorch 1.6 warning #24

Open MichaelMonashev opened 3 years ago

MichaelMonashev commented 3 years ago

/home/xxxx/.local/lib/python3.7/site-packages/adabound/adabound.py:94: UserWarning: This overload of add_ is deprecated:
        add_(Number alpha, Tensor other)
Consider using one of the following signatures instead:
        add_(Tensor other, *, Number alpha) (Triggered internally at  /pytorch/torch/csrc/utils/python_arg_parser.cpp:766.)
  exp_avg.mul_(beta1).add_(1 - beta1, grad)

Lagom92 commented 3 years ago

I change this.

exp_avg.mul_(beta1).add_(1 - beta1, grad)
exp_avg_sq.mul_(beta2).addcmul_(1 - beta2, grad, grad)

exp_avg.mul_(beta1).add_(grad, alpha = 1 - beta1)
exp_avg_sq.mul_(beta2).addcmul_(grad, grad, value = 1 - beta2)

so, it's working, but actually i'm not sure its correct.