search

cybertronai / pytorch-lamb

Implementation of https://arxiv.org/abs/1904.00962
MIT License
369 stars 49 forks source link

UserWarning: This overload of add_ is deprecated #9

Closed jiang-yuan closed 2 years ago

jiang-yuan commented 4 years ago

UserWarning: This overload of add is deprecated: add(Number alpha, Tensor other) Consider using one of the following signatures instead: add_(Tensor other, *, Number alpha) (Triggered internally at /opt/conda/conda-bld/pytorch_1595661244635/work/torch/csrc/utils/python_arg_parser.cpp:797.) expavg.mul(beta1).add_(1 - beta1, grad)

ddelange commented 4 years ago

Might also be the case for addcmul_ and other operations further down the line https://github.com/cybertronai/pytorch-lamb/blob/5ef3ebd5e32f7a7bdcddbb2ce55879bfa88f6a5f/pytorch_lamb/lamb.py#L96-L123

Docs

https://pytorch.org/docs/stable/tensors.html#torch.Tensor.add

mgoldey commented 3 years ago

It looks like #11 resolves the issues with .add_. Is the invocation for addcmul_ actually different from https://pytorch.org/docs/stable/generated/torch.addcmul.html? It doesn't look wrong from a quick glance, and I'm not seeing any errors locally

ddelange commented 3 years ago

Ah, I'm not sure!

My comment was just a breadcrumb FYI, as reaction to some warnings on my screen while using this lib :)

Some other ppl getting these warnings: https://github.com/pytorch/pytorch/issues/32861