Upscale speedup

[AlexZhurkevich]

Hello everybody, I would like to take a binary black and white mask, upscale it to needed resolution and save it as .tif in order to work with it in openslide, as fast/efficient as possible . Past issues were very helpful on this matter, but I thought maybe there is something else that can further speed up the computation or maybe I've missed something. Mask res is: 2614x2108, upscaled .tif should be 41831x33721. My code:

image = pyvips.Image.thumbnail("my_mask.jpg", 41831, height=33721)
image.tiffsave("x.tif", tile=True, compression="jpeg", bigtiff=True, pyramid=True)

It takes --- 15.843008279800415 seconds --- to finish the execution on a 6 core/12 thread CPU. Maybe there is another recommended format that is faster in creation and is also readable by openslide, any recommendations to speed up the computation are welcomed. Also, my_mask.jpg is a black/white image but is saved as a 3 channel uint8 image, maybe there is a way to read thumbnail as grayscale, that might cut some time. Thank you for your time, I appreciate the help.

[jcupitt]

Hi @drhyde488,

Yes, saving a mono image will be quicker. You could also scale up with simple pixel doubling, if blocky output is acceptable.

#!/usr/bin/python3

import sys
import pyvips

# 22s real, 1m31s cpu
image = pyvips.Image.thumbnail(sys.argv[1], 41831, height=33721)
image.tiffsave(sys.argv[2])

# 14s real, 1m20s cpu
image = pyvips.Image.new_from_file(sys.argv[1], access='sequential')
image = image.colourspace('b-w')
image = image.thumbnail_image(41831, height=33721)
image.write_to_file(sys.argv[2])

# 9.6s real, 10.6s cpu
image = pyvips.Image.new_from_file(sys.argv[1], access='sequential')
image = image.colourspace('b-w')
image = image.zoom(1 + 41831 / image.width, 1 + 33721 / image.height)
image = image.crop(0, 0, 41831, 33721)
image.write_to_file(sys.argv[2])

[AlexZhurkevich]

Thank you so much @jcupitt ! Unfortunately it looks like I need to stick to image.tiffsave("x.tif", tile=True, compression="jpeg", bigtiff=True, pyramid=True) any time I'd like to save an image, otherwise openslide treats it like an Image.PIL object and gives me this error:

error: <class 'PIL.Image.DecompressionBombError'>
Image size (1410043615 pixels) exceeds limit of 178956970 pixels, could be decompression bomb DOS attack.

I can resolve it by Image.MAX_IMAGE_PIXELS = 1410043615 but then it loads the whole image into memory which causes a crash since it is too big to fit. Replacing image.write_to_file(sys.argv[2]) with image.tiffsave("x.tif", tile=True, compression="jpeg", bigtiff=True, pyramid=True) in your 14s real, 1m20s cpu does not give any speedup at all. However when trying the fastest one 9.6s real, 10.6s cpu with replacement of with image.tiffsave("x.tif", tile=True, compression="jpeg", bigtiff=True, pyramid=True) it is twice as fast! I was about to be done with it but unfortunately for my use case it looks like it doesnt work properly, let me explain. My end goal is to tile the 41831x33721 mask with the tile size of 512 and count how many white tiles there are:

#Get DeepZoom tile by level and address:
mask_tile = dz_mask.get_tile(mask_level, address)
#Check how white the image is:
mask_extrema = mask_tile.convert('L').getextrema()
#If its 100% white, increase the counter:
if mask_extrema == (255, 255):
   count += 1

It seems because of pixel doubling additional white regions are introduced to the big mask at creation time using the fastest method. For example first two examples you've showed are consistent and count is 1171 tiles, with pixel doubling (fastest) count is 1293, which is just not accurate upscaling.

Do you have any other/additional ideas that might speed up the upscaling process using image.tiffsave("x.tif", tile=True, compression="jpeg", bigtiff=True, pyramid=True) as a save function but also be accurate? Thank you for you time.

[jcupitt]

Hello again, I was saving to a pyramidal tiff. Here's the program again with some timing code:

#!/usr/bin/python3

import sys
import time
import pyvips

start = time.time()
image = pyvips.Image.thumbnail(sys.argv[1], 41831, height=33721)
image.write_to_file(sys.argv[2])
end = time.time()
print(f'thumbnail took {end - start:.2f}s')

start = time.time()
image = pyvips.Image.new_from_file(sys.argv[1], access='sequential')
image = image.colourspace('b-w')
image = image.thumbnail_image(41831, height=33721)
image.write_to_file(sys.argv[2])
end = time.time()
print(f'mono + thumbnail took {end - start:.2f}s')

start = time.time()
image = pyvips.Image.new_from_file(sys.argv[1], access='sequential')
image = image.colourspace('b-w')
image = image.zoom(1 + 41831 / image.width, 1 + 33721 / image.height)
image = image.crop(0, 0, 41831, 33721)
image.write_to_file(sys.argv[2])
end = time.time()
print(f'mono + zoom took {end - start:.2f}s')

You can run it like this:

$ ./upscale.py mask.jpg x.tif[compression=jpeg,bigtiff,tile,pyramid]
thumbnail took 20.37s
mono + thumbnail took 14.24s
mono + zoom took 10.54s

If you put the save options in the filename you can experiment with save settings without needing to change your code.

Have a look at the output of the zoom version in an image viewer and you'll see why you get different numbers of white areas.

Why do you need to count the white tiles in deepzoom output? Maybe there's something that could be changed there?

[AlexZhurkevich]

@jcupitt Ok, I see know. Thank you for a detailed explanation, I am new to pyvips, didn't know I could do it this way. My output looks like this:

thumbnail took 15.01s
mono + thumbnail took 15.30s
mono + zoom took 7.82s

Not sure why mono + thumbnail is not faster than vanilla, even slower.

I simplified the problem, the actual goal is this. I have a .svs slide and a mask, white regions of the mask signify tissue that is useful based on (https://github.com/choosehappy/HistoQC), so accuracy of the upscaled mask is very important. HistoQC saves finalized black and white mask using openslide's get_thumbnail function, hence a much smaller resolution compared to the slide itself. So my logic is: upscale the mask to the resolution of the slide using pyvips, get tiles for both slide and mask at the same time like so:

#Get tiles 
slide_tile = dz_slide.get_tile(slide_level, address)
mask_tile = dz_mask.get_tile(slide_level, address)

#Check if mask tile is 100% white and save slide tile
mask_extrema = mask_tile.convert('L').getextrema()
if mask_extrema == (255, 255):
     slide_tile.save(outfile)

and save slide tile only if the mask tile from the same level/address is completely white.

Previously I was trying to do the same thing with the mask of the original small size wrapping it into openslide object by first opening it through PIL, but the problem is that DeepZoom levels, hence columns and rows of slide and mask will never match since the size (mask dimensions vs slide dimensions) differs based on float numbers and not integers that DeepZoomGenerator takes.

For example: slide dimensions are 41831x33721, mask dimensions are 2614x2108, 41831/2614 = 16.002677888293803, while 33721/2108 = 15.996679316888045, in order for this to work you need to downscale your slide tile size from 512x512 to 31.9946451196x32.0066427448 for a mask, which wont happen because DeepZoomGenerator takes only integers as tile_size, rounding introduces mismatch between cols and rows of slide and mask, mismatch gets bigger the bigger the dimension difference is. As a result the only way I see to combat this problem is to just upscale the mask to the dimensions of the corresponding slide and tile them at the same time solving this problem.

Let me know if you have a better solution than mine, all suggestions are welcomed!

I'll play around with the fastest zoom implementation to make it accurate if possible, otherwise Ill just use vanilla if it is the best in terms of speed and accuracy.

Thank you so much @jcupitt