Registration with simple image rotation doesn't work

[asmagen]

I have tried applying the following command !pathflow-mixmatch register_images --im1 ndpi_images/Li101NCD3.png --im2 ndpi_images/Li101NCD68.png --fix_rotation False --output_dir ndpi_images --gpu_device 0 --transform_type similarity --lr 0.01 --iterations 1000 --min_object_size 50000 --no_segment_analysis True --black_background True to register two images where one of which is rotated 180 degrees relative to the other, hoping it would be a relatively easy task for the registration, but please see below the rather odd result, where the first image didn't change (probably because it's the reference, but the color have shifted from blue to red) and the second hasn't moved at all but was only cropped on the top left. What could the issue be?

Thanks

[jlevy44]

Before performing alignment procedures, we recommend rotating into the proper orientation; while we are building rotation detection mechanisms, you can also try running mixmatch with the segment analysis (which will likely fail on the shape matching part, but does have orientation procedures that we are still modifying) or trying to run our experimental rotation detection function (we are updating the rotation detection algo).

11

14

10

https://github.com/jlevy44/PathFlow-MixMatch/blob/master/pathflow_mixmatch/cli.py#L258

Before we fully implement rotation detection, we recommend sensible use of the mixmatch algorithm, as outlined in our manuscript, where some effort should be taken to achieve similar orientation between the images.

[asmagen]

My images don't come in such extreme orientation inconsistency, I rotated the image to do a basic test of the registration under the assumption that rotation is one of the components of any registration procedure (I'm not referring to the mix-match separate alignment/rotation procedure). Isn't it? Or are you saying it is just for not extreme rotation angle? @jlevy44

[asmagen]

Another major issue to consider now that I think about orientations, what happens if it deviates by 45 degrees or so? Would that need to be recognized computational and adjusted? I'm not sure that's easy for non strand like tissues (what if it's just circular). And what if the dimension scaling that we apply to make the images match the same size will introduce stretching artifacts in a case like a elongated rectangular image which is rotated by some factor and then scaling to the whole width of the reference image will render it very stretched? @jlevy44

[asmagen]

I tested this today with only 100 degrees rotation in one of the images and the result is peculiar as you see here, with trimming of the registered image.

@jlevy44 @sumanthratna