Affine Warp

[sergiumocanu]

Hi there,

First, thank you so much for these resources. They are really great tools.

I was testing the Spatial Transformer layer and have noticed an odd behaviour. In some experiments I have ran, I have noticed an odd behaviour, where when doing a rotation about the z-axis, the image is rotated, however, it's also scaled down. I dug in the code and I noticed the _single_aff_to_shift function. I see that you add the identity matrix to affine matrix.

To remedy that, I subtracted the identity matrix and it worked fine.

I saw the comment in the code, but it still doesn't make sense for me. Maybe I don't understand something, but why did you add this initially?

Kind Regards

[adalca]

Hi @sergiumocanu ,

We have this convention to be consistent with the dense version of the transform, where what we expect is the displacement field, that is, the is the difference from identity, which is often used in these networks. To be consistent with this, we expect the affine to be the difference from identity -- so passing in a matrix of 0s would represent 'no change').

I agree there is some ambiguity here as to what is the best representation. We tried to clarify this in the documentation, but we'd welcome suggestions in general:

    The Layer can handle both affine and dense transforms. 
    Both transforms are meant to give a 'shift' from the current position.
    Therefore, a dense transform gives displacements (not absolute locations) at each voxel,
    and an affine transform gives the *difference* of the affine matrix from 
    the identity matrix.

[sergiumocanu]

I see, ok. Thank you for the response. It makes sense now since the layer is handling both affine and dense warp. I think I missed this particular comment you mentioned.