PatchGenerator does not support certain image sizes

smistad / FAST

A framework for high-performance medical image processing, neural network inference and visualization

https://fast.eriksmistad.no

BSD 2-Clause "Simplified" License

433 stars 101 forks source link

PatchGenerator does not support certain image sizes #171

Open andreped opened 1 year ago

andreped commented 1 year ago

Describe the bug When attempting to run inference with a ResNet50 patch-wise classifier using FastPathology, it was observed that image size 299 x 299 was not supported.

According to the source code, FAST does not support image patches that are not divisible by 2 (see here).

System:

OS and version: [e.g. Windows 10/Ubuntu 18.04] Ubuntu 20.04
GPU and OpenCL vendor (NVIDIA, AMD, Intel) NVIDIA

Expected behavior The PatchGenerator PO should support "any" input shape

cc: @markusdrange

smistad commented 1 year ago

This is not really a bug. TIFF requires the tile size to be a multiplum of 16. TIFF is used in the PatchStitcher when creating a segmentation ImagePyramid on the fly. The TIFF tile size could be different to the patch generator size, but this would lead to very inefficient stitching.

andreped commented 1 year ago

Hmm, I understand. Perhaps we should add a disclaimer regarding this, either in FAST or in FP to inform users about this, as I believe there will be people that may run into the same issue. 299x299 is common for ResNets and Inception backbones.

@markusdrange for now I would just use 256x256 input to the ResNet50 classifier when creating the network. Be sure to change the input layer of the network to use the appropriate image size. In this case, doing resizing in the Compose is no longer necessary.

smistad commented 1 year ago

Note that you can have a patch generator which generates patches at 304x304 (which is a multiplum of 16), and a network which requires 299x299. FAST will then automatically resize the 304x304 patches to 299x299. You should then set the resize back to normal size option in the SegmentationNetwork.

This might be solved more smoothly with adding some overlap automatically to satisfy both constraints, so leave this issue open for now and I can take a look when I have time.