apply_affine does not correctly transform images for scales<1

[andreasmarnold]

When trying to transform images that have a scale <1, the resulting transformed image is wrong. Is it possible that this is a result of defining the affine in relation to the reference image in apply_tf (line 66)?

Here's an example to quickly reproduce the problem. However, it will be necessary to use the plugin.

import napari
from skimage import transform
from skimage.data import astronaut

# Create reference image
reference_image = astronaut()
# Create moving image by resizing reference image and rotating it by 90 degrees
moving_image = transform.rotate(reference_image, 90, resize=True)
moving_image = transform.resize(moving_image, (1024, 1024), order=2)

# open viewer and add images with arbitrary "nonsense" scale < 1
viewer = napari.Viewer()
viewer.add_image(reference_image, name='ref', scale=(0.16, 0.16))
viewer.add_image(moving_image, name='mov', scale=(0.02, 0.02))

# Use affinder to find affine transformation and then apply it to the moving image

# The transformed image contains nothing of the original image but is the right shape
print(f"shape of transformed image: {viewer.layers[-1].data.shape}")
print(f"scale of transformed image: {viewer.layers[-1].scale}")

[jni]

Ah. No, the issue is that we don't take into account the other transformations in the reference image (scale, translate, rotate, shear). Those are defined in addition to the affine.

Thanks for the report!

[andreasmarnold]

Just noticed that I again did not consider rotation and shear in my changes.

Also... when thinking about this, I realized that affine transformations (e.g. translation & rotation) are not inherently commutative. Since I don't know exactly in which order napari applies the transformations, I was wondering what's the best way to consider them here.