Equivariance and Image Resolution

[kristian-georgiev]

Hi Gabriele, thank you very much for the great library!

Is equivariance guaranteed for images of arbitrary sizes/resolutions? It seems like equivariance is satisfied only for certain sizes. As an example, the C4-invariant 10-4 Wide ResNet defined and tested in examples/e2wrn.py seems to only be invariant to 90-degree rotations for images with widths of the form 4k+3.

To exemplify this, the output of this snippet

m = Wide_ResNet(10, 4, 0.3, initial_stride=1, N=4, f=False, r=0, num_classes=10)
m.eval()
for img_size in range(30, 50, 1):
    x = torch.randn([1, 3, 30, img_size])
    xrot = torch.rot90(x, k=1, dims=[2, 3])
    with torch.no_grad():
        z = m(x)
        zrot = m(xrot)
        print(f'Image size is 30x{img_size}. Equivariant:' +\
              ('YES' if torch.allclose(z, zrot, atol=1e-6) else 'NO'))

is

Image size is 30x30. Equivariant: NO
Image size is 30x31. Equivariant: NO
Image size is 30x32. Equivariant: NO
Image size is 30x33. Equivariant: YES
Image size is 30x34. Equivariant: NO
Image size is 30x35. Equivariant: NO
Image size is 30x36. Equivariant: NO
Image size is 30x37. Equivariant: YES
Image size is 30x38. Equivariant: NO
Image size is 30x39. Equivariant: NO
Image size is 30x40. Equivariant: NO
Image size is 30x41. Equivariant: YES
Image size is 30x42. Equivariant: NO
Image size is 30x43. Equivariant: NO
Image size is 30x44. Equivariant: NO
Image size is 30x45. Equivariant: YES
Image size is 30x46. Equivariant: NO
Image size is 30x47. Equivariant: NO
Image size is 30x48. Equivariant: NO
Image size is 30x49. Equivariant: YES.

This seems like an artifact caused by dilation/pooling but I am not sure whether there's any other factor causing this behaviour. Is this the case?

Lastly, would we observe a similar behaviour if we are working with field types composed of irreps of SO(2)? I don't have an intuition on how dilation is going to interact with guaranteeing equivariance there.

Thanks in advance!

[Gabri95]

Hi @kristian-georgiev

Thanks for your message! This is indeed an artifact of the pooling and dilation. Using SO(2) equivariance is not going to solve this, unfortunately.

Have a look at page 3 here: https://arxiv.org/pdf/2004.09691.pdf We tried to explain a bit this issue and how to solve that there. Let me know if this helps

Best, Gabriele

[kristian-georgiev]

This makes perfect sense, thank you!