Logics in function "upsample_preds" hard to understand

[TWJianNuo]

Thanks for your amazing work!

While trying to utilize this amazing work, I can not understand on implementation in the code.

In funciton "upsample_preds" (used only in function match), the final flow is further refined using "self.conv_refiner", however, the estimated residual flow is not added over the original final flow. Instead, a re-sampling is executed.

Related Codes:

query_coords = torch.meshgrid((
        torch.linspace(-1 + 1 / h, 1 - 1 / h, h, device="cuda:0"),
        torch.linspace(-1 + 1 / w, 1 - 1 / w, w, device="cuda:0"),
    ))
query_coords = torch.stack((query_coords[1], query_coords[0]))
query_coords = query_coords[None].expand(b, 2, h, w)
delta_certainty, delta_flow = self.conv_refiner["1"](query, support, flow)
displaced_query_coords = torch.stack((
        query_coords[:, 0] + delta_flow[:, 0] / (4 * w),
        query_coords[:, 1] + delta_flow[:, 1] / (4 * h),
    ),dim=1,)
flow = F.grid_sample(flow, displaced_query_coords.permute(0, 2, 3, 1))

[Parskatt]

It seems that you found a minor bug! At a previous point we used the grid_sample approach to regularize the upsampled flow, but it should not be in the implementation here.

I will have a look at this (using the actual displacement), and get back to you :)

[Parskatt]

@TWJianNuo Thanks for finding this. Some more updates to the inference might come in the following weeks, I'll keep you posted :)