libvips / pyvips

python binding for libvips using cffi
MIT License
640 stars 49 forks source link

Unable to call VipsForeignSaveTiffFile #512

Open afriebel85 opened 1 day ago

afriebel85 commented 1 day ago

For my Akoya multiplex wholeslide sections I performed spectral unmixing with the Akoya InForm software which generated for a 8-channel scan 240 patches with float pixel precision resulting in a total of 166GB for this one scan. I now want to stitch these patches back together, and downcast the pixel precision to ushort. I started with the very nice script from Ajay Zalavadia, used the first 8 pages as they represent the 8 channels at highest magnification (see below) and added the downcasting to ushort (after multiplication with 10) in the interleaved_tile method. I tested the method on a smaller subset of 17 patches, and it took ~20min to complete. I checked the resulting ome.tif in QuPath and everything looked fine. I then tried to stitch the full set, however I got a crash after about 20 minutes:

2024-10-18 14:18:38,846 - WARNING - VIPS: error in tile 0 x 20992
2024-10-18 14:18:38,862 - WARNING - VIPS: error in tile 0 x 20480
2024-10-18 14:18:39,671 - INFO - VIPS: threadpool completed with 8 workers
2024-10-18 14:18:39,908 - ERROR - Error saving OME-TIFF: unable to call VipsForeignSaveTiffFile
source input: Using code not yet in table
source input: Using code not yet in table

I observed RAM usage, which shouldn't be a problem, it remains at ~3.5GB. Disk space isn't a problem as well with 1.3 TB available. I tried to diagnose the patches, in case there are faulty ones, but all can be read with tifffile and seem to have the exact same setup like so:

2024-10-18 20:27:41,799 - INFO - Page 0: Size: 5580 x 4188, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,799 - INFO - Page 1: Size: 5580 x 4188, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,799 - INFO - Page 2: Size: 5580 x 4188, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,799 - INFO - Page 3: Size: 5580 x 4188, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,799 - INFO - Page 4: Size: 5580 x 4188, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,799 - INFO - Page 5: Size: 5580 x 4188, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,799 - INFO - Page 6: Size: 5580 x 4188, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,800 - INFO - Page 7: Size: 5580 x 4188, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,800 - INFO - Page 8: Size: 348 x 261, Pixel Type: uint8, #Bands: 3, Color Space: 2
2024-10-18 20:27:41,800 - INFO - Page 9: Size: 2790 x 2094, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,800 - INFO - Page 10: Size: 2790 x 2094, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,800 - INFO - Page 11: Size: 2790 x 2094, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,800 - INFO - Page 12: Size: 2790 x 2094, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,800 - INFO - Page 13: Size: 2790 x 2094, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,800 - INFO - Page 14: Size: 2790 x 2094, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,800 - INFO - Page 15: Size: 2790 x 2094, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,800 - INFO - Page 16: Size: 2790 x 2094, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,800 - INFO - Page 17: Size: 1395 x 1047, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,800 - INFO - Page 18: Size: 1395 x 1047, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,800 - INFO - Page 19: Size: 1395 x 1047, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,800 - INFO - Page 20: Size: 1395 x 1047, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,800 - INFO - Page 21: Size: 1395 x 1047, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,800 - INFO - Page 22: Size: 1395 x 1047, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,800 - INFO - Page 23: Size: 1395 x 1047, Pixel Type: float32, #Bands: 1, Color Space: 1
2024-10-18 20:27:41,800 - INFO - Page 24: Size: 1395 x 1047, Pixel Type: float32, #Bands: 1, Color Space: 1

I am currently at a loss at what could cause this problem. What does the error message indicate?

jcupitt commented 1 day ago

Hello @afriebel85,

How many files are you stitching together? Windows has a hard limit of 2000 (I think). You might need to find a linux machine (or use WSL) if you are joining more than that.

You might also be hitting stack size limits. You are doing:

image = black(...)
for tile in many_tiles:
  image = image.insert(tile, ...)

So you are making a pipeline that's as deep as the number of tiles you are stitching. This will also fail for more than a few thousand tiles. You'd need to assemble in sections, then join with something like composite.

What are you doing with overlaps? You could do a feathered join, it might look nicer.

As your code stands, I think you'll write a float OME TIFF. Do you need all that precision? You'd save a lot of disc and CPU with an 8-bit image.

Other than that your code looks reasonable. I think I'd need to be able to run a copy here to understand what's going on in more detail. You'll need to make something you can send me that lets me reproduce the problem.

afriebel85 commented 1 day ago

I got 240 files that need to be stitched together. Each is 5580 x 4188 pixels and 8 color channels with image pyramid caluclated. There is no overlap between the tiles. The pixel precision of these tiles is float, which is what we got from the spectral unmixing software. During the stitching I downcast to ushort pixel type, to save disc space & CPU.

I will try to generate a minimal example, without having you to download 160 GB.. :) Thank you for the quick response.

jcupitt commented 1 day ago

Is this colour deconvolution, or something fancier? If it's just colour deconv, you can do that directly in pyvips and avoid a lot of processing.

If your tiles are in a regular grid, you can use arrayjoin rather than insert in a loop.

afriebel85 commented 1 day ago

As far as I understand, the Akoya system captures the full fluorophore spectra, instead of only narrow bands, during imaging and then unmixes the different spectra, thereby greatly improving the SNR and isolating autofluorescence. I am not sure which exact method the Akoya software is using, but in general it can be achieved using algorithms like linear unmixing or non-negative matrix factorization. In case you are interested, here is a their appnote.

I'll do some further experimenting with the code I have and try to assemble a minimal failing example for sharing.

jcupitt commented 1 day ago

Yes, that sounds like colour deconvolution. There's some sample pyvips code here:

https://github.com/libvips/pyvips/issues/289#issuecomment-994539912

You just need to generalise to more than three channels, it should be easy. Maybe for another time!