Pytorch 1.6 autocast - Githubissues

NVlabs / pacnet

Pixel-Adaptive Convolutional Neural Networks (CVPR '19)

https://suhangpro.github.io/pac/

Other

511 stars 79 forks source link

Pytorch 1.6 autocast #21

Open prottmann opened 4 years ago

prottmann commented 4 years ago

Are there any plans on adding support of autocasting of Pytorch 1.6? Currently the weights and kernels are sticking to float and the gradients are half which give a type missmatch error in backpropagation.

For running my example i'm using the code of the 1.4 branch.

jsetty commented 2 years ago

@prottmann Did you happen to find a work-around? I am trying to use this in my model and I am facing the same error with Float and Half.

sttzr commented 1 year ago

Setting PacConv2d(..., native_impl=True) resolves this issue for me. This uses a different implementation which is "using only standard pytorch layers/operations" and those already support autograd/autocast. It might have a bit higher peak memory usage, as pointed out by the author. Note that I'm using the th14 branch