Misalignment with TensorFlow weights

[ncherel]

Hello, First of all, thank you for your great work.

I have observed a strange problem when using the converted weights from TensorFlow (thanks again!). I do not get exactly the same results than the official implementation. They are very close but slightly misaligned:

The coarse network seems to do its job correctly but something happens during the refinement step. Do you have an idea of where it could come from ?

Tested with PyTorch 1.7 and TensorFlow 1.15

[daa233]

@ncherel Thansk for your attention!

I just confirmed that there is a mislalignment for the output. Thanks for pointing out it happens in the refined stage.

The resullt of examples/imagenet/imagenet_patches_ILSVRC2012_val_00025892_input.png by the converted TF model:

Sorry, I am not sure about the reason for this problem and have little time to find it recently.

Here are some possible reasons:

• Different conv padding (or other parameters) misalignment between TF and PyTorch.
• Different implementation details in the contextual attention module between TF and PyTorch.

By the way, the TF model is converted by @zphang semi-manually in this pr, you may try to convert it by yourself and check it again.

[ncherel]

Thanks for your input. I finally managed to reproduce the official results using your code. I had to change a few things :

• convert input image from RGB to BGR
• add a same_padding operation before the conv_downsample and slightly modify the function. When the total padding is odd, your code will insert 1 more for left / top rather than right / bottom as TF
• last, Pytorch interpolate can't align corners when using nearest neighbors. Switching off corners alignment in TF gives the same result as this PyTorch code. Maybe one day PyTorch will support this and everything will be 1:1 equivalent.

[daa233]

Awesome! Could you please share your code in a branch/pull request so that other people may learn from it? Thanks for your excellent work!