scaling transformation for image with different shape

[batukav]

Hello @cdgatenbee ,

Thank you for making this great public. I've found valis very useful and easy to use.

During the registration, I am facing the warning:

UserWarning: scaling transformation for image with different shape. However, without knowing all of other image's shapes, the scaling may not be the same for all images, and so may not overlap.

Input is two .svs images and I used a fresh Valis installation (1.0.0rc15). The input images have the size ~50k x 35k. I am using the command

registrar = registration.Valis(slide_src_dir, results_dst_dir, imgs_ordered=True, reference_img_f= './input_images/ref.svs', max_processed_image_dim_px = 960, max_image_dim_px = 2500, align_to_reference = True, crop = 'all') and can confirm that the warning is independent of the used crop option.

I managed to track down the issue to the finalize function in registration.py. In my case, the processed images had the size 655 x 960 before finalize function and they got changed to 656 x 969 after finalize . I think this triggered the UserWarning.

I am not sure how much this will affect the overall results and would it be just safe to ignore the warning. What could be the reason of this behavior and how might we can fix this?

[teacakedeadlift]

I get this warning as well and it seems to depend on the slides used as input. Using the same code but missing first and last slides of the stack from the ./slides directory results in the warning, but when using all of them I get no warning. Incidentally the max_image_dim_px is reset to different values as well - 539 in the first case, 497 in the second. Assume this may have something to do with it, but don't know the reasoning behind it, or a fix (if one is needed).

[cdgatenbee]

Hi, sorry I hadn't gotten around to answering this earlier. @teacakedeadlift is correct that the warning message is thrown when the images valis is trying to warp have different shapes than the ones it used during registration (i.e. after pre-processing and downscaling), or if not all of the other images are available. This is because valis has re-estimate the size of the output image, but it's possible this new shape wouldn't be the same for other images even if the other images were scaled similarly (although this can be done when all of the images are present). However, this should only affect warping scaled down versions of the slide, and won't alter what happens when you warp your full size image.

Despite the above, I'm not sure why this warning would be coming up during the finalize step, and so I'll look into it. @batukav, could you tell me, is there a particular reason you have set max_processed_image_dim_px = 960 and max_image_dim_px = 2500? It could help me in the debugging process.

Best, -Chandler

[batukav]

Hi @cdgatenbee.

max_processed_image_dim_px = 960 worked the best for my dataset. I don't have a particular reason to use max_image_dim_px = 2500 but I am also not very sure about what this parameter is used for. Should I try with a different value?

[cdgatenbee]

Hi @batukav,

Thanks for the extra info. Τhe parameter max_image_dim_px determines the size of the images that each Slide object would warp by default when you call Slide.warp_img, which you may want to do if you're going to generate images for a figure, presentation, etc... It doesn't actually impact the registration (max_processed_image_dim_px does, though), nor have anything to do with warping the full resolution WSI. So its main effect is in determining the size of the images for your figures, and the size of the saved Valis object (e.g. the registrar.pickle object in the data folder). A value of 2500 seems reasonable, since it should provide decent resolution and not take up too much space.

Best, -Chandler

[teacakedeadlift]

To add to this I ran 2 alignments with identical scripts on the same slide stacks, but just removing the first and the last slides for the second run as they were the worst aligned (of 14 slides). The warning only occurred on the second run with 12 slides, but the alignments for these 12 were marginally worse than that for the identical slides in the stack of 14. This is reflected in the errors non_rigid_D and non_rigid_rTRE being marginally worse (except for the 1st and 14th outliers), and in manual inspection of the warped and saved slides. I'm not sure how bad it was as comparing the 2 manually is difficult due to the size of the slides so some areas looked fine and others were out. Sadly, the errors in both (due to quality of slides and range of changes across sections rather than the alignment) mean accurate checking is impossible, and both alignments were usable. I just thought it was odd behaviour.

Thanks

Phil

[cdgatenbee]

That is indeed odd behavior, @teacakedeadlift. I'm not really sure why the warning would only be thrown when you're using a subset of slides. As far as the 12 slides producing worse results than the 14 slides, is valis putting the 12 in the same order as when you use all 14? If they get ordered differently, maybe that would explain the marginal difference?

[teacakedeadlift]

I'll double check tomorrow if I have time but yes warning only when I omit slides, and marginally lower errors. I should clarify that those two slides are HE while the others are H-DAB so maybe during normalisation something happens which later impacts alignment. Does normalisation occur pairwise? The ordering is the same as I don't change numbering, just move slides into a subdirectory within the slides directory so valis doesn't see them (just warns a slide object can't be found) - it just means it starts with 2- for the second run.

This may be related to some other errors I get when I include HEs (from memory an OpenCV related one) that disappear once I remove them and just use H-DAB. I originally thought it was due to the landmarks being too different but it happens for one case even with low resolution alignment (500 for rigid and non_rigid).

I'll double check and maybe do another test run with smaller subset to see what happens to the errors.

Phil

[cdgatenbee]

@teacakedeadlift, I think you might be right that the normalization step is causing the difference between use 12 and 14 slides, as the images are batch normalized. So, removing 2 images changes the target image statistics, and after pre-processing, the images in the set of 12 probably look a little different than the images in the set of 14. Perhaps that's why there is a small difference in error?