search

NHPatterson / napari-wsireg

Whole slide image registration through wsireg in napari.
BSD 3-Clause "New" or "Revised" License
11 stars 3 forks source link

wsireg registration appears to fail at final output step: "Killed" #3

Closed KidElectric closed 2 years ago

KidElectric commented 2 years ago

Hello, I am using the napari-wsireg plugin installed today via pip. I am registering some large sequential section H&E .svs WSIs (and multimodal WSIs in the future). So this package is very exciting to me!

The process appears to go pretty well (WSIs are preprocessed in the cache, and I see 10 successful iteration plots and txt files in the output folder, using rigid->affine). However right at the end it saves a size 0kb ome.tiff file. The output says: Time spent on saving the results, applying the final transform etc.: 1 ms. using default compression saving to /[...]test-1_to_0_registered.ome.tiff transforming : 0 transformed : 0 transforming : 1 transformed : 1 transforming : 2 transformed : 2 Killed

Where should I start for debugging this? Thank you!

NHPatterson commented 2 years ago

Oof, not a very informative error message! I'm guessing this is a segfault in ITK (used to manage images/transformations) and an out-of-memory error when saving a very large RGB plane. wsireg will save RGB data interleaved, where each RGB band lives next to one another: image This diagram shows the binary data layout for RGB vs planar (typical of fluorescence microscopy).

Internally, each color in RGB is transformed individually and held in memory to compose them interleaved (in memory again) for writing to OME-TIFF. This has been something I want to optimize and I have some thoughts on how to address it so I will try to sort in the next couple of days. I'm making some changes to the wsireg codebase that I think will help and will write back, hopefully will wrap up tomorrow with some fixes and improvements that will help here.

KidElectric commented 2 years ago

Thank you for this detailed answer! I have access to HPC resources, do you imagine using a lot more RAM would help here? Is there a way to estimate the RAM needed from the size of the images... like say a 4gb .svs file -> 40gb uncompressed in memory, and 2 of those = 80gb+ 10-20gb overhead for models etc? Just wondering if there is a way to ballpark to request enough RAM to test this idea!

KidElectric commented 2 years ago

and I am not an expert in how to design this stuff but I wonder if any of the image batching architecture from CNN pipelines could be repurposed here (pytorch etc).

KidElectric commented 2 years ago

I will say that running with a lot of RAM seems to have solved the problem. I didn't debug much, just set it to 256gb. Closing this out for now! Thanks for your help and I look forward to your updates!

NHPatterson commented 2 years ago

Those are great points thanks. Some of those things wsireg kind of does already but not by default (there's another writer that writes the WSI tile-by-tile after registration but it's still a bit experimental, but I'll expose it in the GUI in the next release). Glad that increasing RAM helped. I hope the registration was successful too!

For RAM estimates, you should only need the uncompressed size of your target image and some overhead. It sounds like you have some big images at 40 GB uncompressed. This might be a good feature to estimate RAM consumption before registration proceeds.

KidElectric commented 2 years ago

The registration is beautiful! I tried rigid->affine and rigid->affine-> nl . Both worked and the rigid->-affine->nl registration is pretty remarkable. I guess the real test will be the full stack of 80 sequential WSI sections of which I've just been debugging on the first 2. I will be very curious to see how far I can get! Thank you for this great tool!