Closed ProGamerGov closed 4 years ago
In-place operations are generally slower, often end up requiring more memory, and can interfere with Autograd. Thus, they should always be avoided unless you have an edge case that requires them.
https://pytorch.org/docs/stable/notes/autograd.html#in-place-operations-with-autograd
Lucid also doesn't do an in-place operation either: https://github.com/tensorflow/lucid/blob/master/lucid/optvis/param/spatial.py#L86
Oh I didn't know that. Thanks!!
In-place operations are generally slower, often end up requiring more memory, and can interfere with Autograd. Thus, they should always be avoided unless you have an edge case that requires them.
https://pytorch.org/docs/stable/notes/autograd.html#in-place-operations-with-autograd
Lucid also doesn't do an in-place operation either: https://github.com/tensorflow/lucid/blob/master/lucid/optvis/param/spatial.py#L86