search

frankkramer-lab / MIScnn

A framework for Medical Image Segmentation with Convolutional Neural Networks and Deep Learning
GNU General Public License v3.0
407 stars 116 forks source link

Clarification? #134

Closed innat closed 1 year ago

innat commented 2 years ago

I've some queries regarding the approach in the code.

  1. HERE, you set analysis="patchwise-crop", patch_shape=(160, 160, 80). What does this analysis param exactly mean? And what is the relation with the patch_shape param? Does it mean that the 3D volume will be randomly cropped-and-resize? But HERE, in the docstring, it's mentioned that

patch_shape: ... This parameter will be redundant if a full image or patchwise-crop analysis is selected!!

Following this, what is the role of this program?

  1. The training data, comes with variable depth size, i.e., 301, 200, 39, 66, 418 etc. The lowest size is 39. With that how did you set input with depth == 80 here?
innat commented 2 years ago

@muellerdo

muellerdo commented 2 years ago

Hello @innat,

sorry for the late reply.

  1. Thanks for pointing that out! The docs here are in fact incorrect / misleading :o

Normally the analysis types works like the following:

For larger volumes like in ct scans, the patchwise-crop strategy with an adequate resampling is quite effective for model training and currently the state-of-the-art in challenges.

What does the 'pp.patchwise_overlap' parameter do? This one is only applied on a patchwise-grid strategy. But have in mind that patchwise-crop uses also patchwise-grid for inference! The point is that segmentation predictions for the pixels on edges of images/patches are always a bit wacky / uncertain / non-stable. Thus, patching approaches, especially approaches with smaller patch_shapes, result into a lot of pixels with a higher uncertainty -> we end up with a edge bias in our predictions after combining the patches back together to a complete image for the inference. The avoid / reduce this bias, overlapping patches are created. This allows to have multiple predictions for a single pixel on the edges which gives these edge pixels more certainty of their prediction.

  1. With the data_exploration script in this repo, I identified the overall distribution of the depth axis after resampling and thought that for nearly all cases, a depth size of 80 is sufficient. You can also go higher + padding, depending how much VRAM you GPU has, but going too small will probably lead to performance reduction.

You are right, that in this dataset a few samples had less than 80 after resampling on their depth axis. I have to take an assumption (but I'm 75% sure) that our novel patching interface, automatically pads volumes to the minimum patch_shape. If this is not the case in the provided MIScnn version in this repo, please notify me. Then, we have to add an additional Padding Subfunction to the subfunction_list to ensure the minimum shape / patch_shape.

Hope that I was able to help you.

Cheers, Dominik

PS: I think this would be helpful also for other MIScnn users. Would it be ok for you, if we move this issue to the MIScnn repo?

muellerdo commented 2 years ago

PSS: I committed an enhancement for the misleading doc on the MIScnn dev branch.

innat commented 2 years ago

@muellerdo Thanks for the reply.

patchwise-grid -> apply subfunctions on the whole image but split the preprocessed image in patches in a grid-like fashion. then pass all patches to the GPU. the patches have the size of patch_shape

About patchwise-crop, it simply means randomly cropping the sample and using it for model input, right? For example:


volume_1 = (512, 512, 300)
volume_2 = (312, 312, 300)
volume_3 = (224, 224, 300)

volume = [volume_1, volume_2,  volume_3, ....]
volume = some_random_crop_fn(volume, target_shape=(150, 150,  80))
volume.shape
(150, 150, 80)
(150, 150, 80)
(150, 150, 80)
...
....
innat commented 2 years ago

patching interface, automatically pads volumes to the minimum patch_shape.

I think it's programmed HERE. To match with the target volume, does it repeat the same slices that times?

innat commented 2 years ago

PS: I think this would be helpful also for other MIScnn users. Would it be ok for you, if we move this issue to the MIScnn repo?

Yeah, sure. (I think it would be fitted better if MIScnn has any discussion tabe in its repo.)

innat commented 2 years ago

I was reading paperwork, which used the 20 cases of covid ct images. While reading the paper, I came across the name of this repo (MIScnn). HERE is the paper. Check page 4, the last sentences of the first paragraph. Does it somehow related to this repo? It states,

... Second, our experiment chooses to train in the MIScnn pipeline, which is mostly used for training on small data sets. Tere are two methods to alleviate the shortcomings of insufficient data

muellerdo commented 2 years ago

About patchwise-crop, it simply means randomly cropping the sample and using it for model input, right? For example:

Correct!

I think it's programmed HERE. To match with the target volume, does it repeat the same slices that times?

That is the general padding subfunction of MIScnn (for the user) ;)

The integrated padding of our patching operations can be seen here:
https://github.com/frankkramer-lab/MIScnn/blob/6a6b6a306137004587159581fb251adbc429668f/miscnn/processing/patching/patch_operations.py#L41

The automatic padding for patches (if required) utilizes a 'minimum' mode, which means that the lowest number of the sample is repeated to fill missing pixels. In a medical context (CT/MRI/ultra sound/...) this is normally the background pixel intensity value.

Yeah, sure. (I think it would be fitted better if MIScnn has any discussion tabe in its repo.)

Thanks for the feedback. I will move this issue as well as open a discussion forum in the repo! :)

I was reading paperwork, which used the 20 cases of covid ct images. While reading the paper, I came across the name of this repo (MIScnn). HERE is the paper. Check page 4, the last sentences of the first paragraph. Does it somehow related to this repo? It states,

Mhm. No, the study of this repo was published as Robust chest CT image segmentation of COVID-19 lung infection based on limited data: https://doi.org/10.1016/j.imu.2021.100681

Even if it is really nice to hear that other people use MIScnn for their research, it is still quite sad to see that some authors miss to cite the framework publication.

muellerdo commented 2 years ago

Transfered to MIScnn repo

innat commented 1 year ago

@muellerdo Hi. Could you please move this issue to the discussion section? I think it better suits there.