keypoint_registration.py image loading

[elmbeech]

Hi @cdgatenbee and @bswhite ,

Very nice implementation! In line 76 and 77 of your keypoint_registration.py code, I wonder why you open the file with pillow (PIL.Image.open) and not use scikit-image for that (skimage.io.imread). Is there a reason or is this just habit?

Elmar

[cdgatenbee]

Thanks @elmbeech! You're right, I think PIL was used out of habit but using skimage.io.imread should work just fine.

[elmbeech]

Thank you for this prompt answer - and the code of course! I will then close this issue.

[elmbeech]

Hi @cdgatenbee and @bswhite ,

It is already a wile ago, but I came back to this code. I have again a question line 76 and 77. What do you try to achieve with this transformation sequence: img_as_ubyte(img_as_float()) Make sure that the loaded image is 8[bit]? I think this is what the code does.
Scale the pixel intensity? If this was the intent, I think this is not what the code does. In any case, why do we have to do this transformation?

Thank you, Elmar

[cdgatenbee]

Hi @elmbeech , Yes, you're correct, the intention is to convert a float image with possible values between 0-1 to a uint8 image with possible values between 0-255. This is because, as far as I know, most (all?) of the feature detectors in OpenCV work on uint8 images. The series of transformations wouldn't be necessary if the image is already uint8. Does that help?

Best, -Chandler

[elmbeech]

Thank you, Chandler for your prompt response!

Yes, that helps. I was not aware that OpenCV works only on unit8 images. But maybe that is the reason that my registration fails ... becasue I have 16[bit] images. I will give a try.

Elmar

[elmbeech]

@cdgatenbee , I have an additional question to the apply_transform function (line 41 following). The if else statement.

In line 82 you build a fresh similarity transformer. This one has classe string <class 'skimage.transform._geometric.SimilarityTransform'>.

When do you expect to have the if case, a <class 'skimage.transform._geometric.PolynomialTransform'>? Because in this code we don't build this transformer.

[cdgatenbee]

Hi @elmbeech, during the workshop we wanted to try out several different scikit-image transformers, and PolynomialTransform was one of them. The PolynomialTransform doesn't use matrices to perform the transform, and so we had to use it differently than the SimilarityTransform. Anyhow, the results from PolynomialTransform weren't great for us, so I'd probably stick with the SimilarityTransform. Hope that helps clear things up.

Best, -Chandler

[elmbeech]

Hi @cdgatenbee,

I just summited a pull request. OpenCV can actually handle 8 and 16 bit images. I changed the code, that this now properly works. Furthermore, I added an image normalization step before the actual registration strep.

By the way, this is where and how we use this registration code:

• https://gitlab.com/bue/jinxif/-/blob/master/jinxif/keypointregist.py
• https://gitlab.com/bue/jinxif/-/blob/master/jinxif/src/template_registration_mscene.py
• https://gitlab.com/bue/jinxif/-/blob/master/jinxif/src/template_registration_mscene.m (original matlab code)

Thank you! This all would not have been possible without yours and @bswhite `s code and explanation. Elmar

[cdgatenbee]

Thanks @elmbeech! I've just merged your updates. Very happy to hear you're finding the code useful :)

I've also been working on a separate Python image registration pipeline that is able to register and save whole slide images as ome.tiff using both rigid and non-rigid transformations. Maybe you'll find it useful as well? if you'd like to take a look, here's a link to the pre-print https://www.biorxiv.org/content/10.1101/2021.11.09.467917v1.full. Hoping to have the code up on github very soon :)

-Chandler

[elmbeech]

Thank you @cdgatenbee !

Actually, I figured out, the code "works", but is not really registering. Attached is a simple test case. A TMA that we try to register over 10 rounds to round 1. In the DAPI QC plot you can see that it is not registering at all (dark blue bands in most of the rounds). I attached as well a QC image (round 6Q) from this keypoint registration code. It looks like skimage.transform.warp not does anything. But maybe this is caused by the cv2.KAZE, cv2.BFMatcher, or cv2.findHomography function. I don't know.

For the moment, I give up, and we keep on registering with matlab. As soon VALIS is out, I am happy to test it, to plug it into our workflow. The concept sound very intriguing, and if it does the trick, we will definitely use it.

You wrote that you will make the code available on github. Do you consider making it VALIS available as pypi package, that it is installable via pip? That would certainly be most portable. I just saw that ashlar is pip installable too. Since we have cyclic IF data, I think I will give a try too. Anyhow, I am looking forward to VALIS anyway.

Best, Elmar (OHSU ChinLab)