The error in forward-propagation

[racso10]

Thanks for providing the code of forward warping operation in pytorch. But I get some errors when I add the forward_warp to a nn.Module. The error occurs in the forward-propagation. Here is the code and the error information.

import torch
import os
import torch.nn as nn

from Forward_Warp import forward_warp

class FW(nn.Module):
    def __init__(self):
        super(FW, self).__init__()
        self.conv = nn.Conv2d(3, 2, 3, 1, 1)
        self.fw = forward_warp(interpolation_mode='Bilinear')

    def forward(self, x):
        flow = self.conv(x).permute(0, 2, 3, 1).contiguous()
        return self.fw(x, flow)

if __name__ == "__main__":
    os.environ["CUDA_VISIBLE_DEVICES"] = '0'

    criterion_train = torch.nn.L1Loss()
    model = FW()

    x = torch.randn(4, 3, 32, 32).cuda().contiguous()
    y = torch.randn(4, 3, 32, 32).cuda()
    model.cuda()
    optimizer = torch.optim.Adam(model.parameters(), lr=0.01)

    for i in range(10):
        y_pred = model(x)
        loss = criterion_train(y_pred, y)

        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        print(loss.item())

Traceback (most recent call last):
  File "Forward-Warp-master/test/test_new.py", line 31, in <module>
    y_pred = model(x)
  File "/data/anaconda3/envs/pytorch1.4/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "Forward-Warp-master/test/test_new.py", line 16, in forward
    return self.fw(x, flow)
  File "/data/anaconda3/envs/pytorch1.4/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/data/anaconda3/envs/pytorch1.4/lib/python3.6/site-packages/Forward_Warp-0.0.1-py3.6.egg/Forward_Warp/forward_warp.py", line 59, in forward
TypeError: save_for_backward can only save variables, but argument 2 is of type int

Moreover, if with torch.no_grad(), the code works. The flow is calculated by CNN, does it causes the problem?

The error is occur both in pytorch1.4 and pytorch1.7, with cuda 10.1 and python 3.6.

[lizhihao6]

It is a bug that has been solved now, try git pull. Thank you for your proposal.

[racso10]

Thanks a lot! But it still has some errors in backward-propagation.

Traceback (most recent call last):
  File "/Forward-Warp-master/test/test_new.py", line 36, in <module>
    loss.backward()
  File "/data/anaconda3/envs/pytorch1.4/lib/python3.6/site-packages/torch/tensor.py", line 195, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/data/anaconda3/envs/pytorch1.4/lib/python3.6/site-packages/torch/autograd/__init__.py", line 99, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: function forward_warp_functionBackward returned an incorrect number of gradients (expected 3, got 2)

[lizhihao6]

Hi, it should be fixed now.