search

google / automl

Google Brain AutoML
Apache License 2.0
6.18k stars 1.45k forks source link

p.add_(torch.sign(update), alpha=-group['lr']) has been made more eff… #1193

Closed sachinspanicker closed 10 months ago

sachinspanicker commented 1 year ago

…icient by using the torch.sign function's inplace mode. This will prevent the need to create a new tensor for the updated parameter, which can save a significant amount of time for large models.

google-cla[bot] commented 1 year ago

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

crazydonkey200 commented 10 months ago

Sorry that I have missed this pull request earlier. Thanks for the contribution!

liuslnlp commented 9 months ago

Error

add_() received an invalid combination of arguments - got unrecognized keyword arguments: inplace

My torch version

>>> import torch
>>> torch.__version__
'2.1.0.dev20230829+cu121'
fbotp commented 8 months ago

@liuslnlp add_ is already an inplace method, so I remove this parameter in method, and it works, you can have a try

yongchanghao commented 8 months ago

As the comments above suggested, there isn't such an argument as inplace=True in torch.Tensor.add_. for both torch versions above or under 2.0. I believe this code hasn't been tested. A rollback might be needed? @crazydonkey200