Update inference script for BRATS

[diazandr3s]

• If multi-sequence, assume all volumes are of the same size
• Intensity normalization should happen after spacing

https://github.com/lassoan/SlicerMONAIAuto3DSeg/assets/11991079/99b3f5d2-48da-4e69-9c3e-eac9bc2a4b21

[diazandr3s]

Hi @che85,

I'm working on 5 more models for BRATS challenge. Unfortunately, we may need to think of another way of customizing the intensity normalization for your custom case. After checking with the Auto3DSeg devs, they've told me the intensity normalization transform should be the last one. This means, after the Spacing transform - This is to make the model more robust to single and multi-sequence.

Happy to further discuss this.

[diazandr3s]

We need to find a way to enable using different modalities and support brats, too.

@lassoan, can we do the same as we did for Prostate model?

[diazandr3s]

Another solution would be to have two separate inference scripts: one for BRATS and the one we currently have?

[lassoan]

BRATS data set has been always an outlier and required special treatment at many places, probably because the authors decided to publish images that went through lossy processing and dumped them into NIFTI images that violate the NIFTI standard. You would not want to ask users to repeat this, but users should always provide their clinical images and the model knows how to use them for inference (crop, pad, resample, normalize, etc.). Therefore, it makes sense to handle BRATS as a special case, isolated from everything else.

The approach you used for the prostate data set and @che85 for cardiac segmentation seemed reasonable to me. The best is if you clarify the details directly with him - maybe it can become a template for doing similar segmentation tasks, or maybe you end up finding a better way to segment such data sets with Auto3DSeg.

[che85]

@diazandr3s Where is the problem with running normalization on separate keys later if different modalities/normalization were used?

[diazandr3s]

Thanks for clarifying all this, @lassoan.

I decided to train these BRATS models and have them available to demonstrate the capabilities of the Slicer+MONAI setup. Also to make the issue with highly preprocessed datasets more visible.

@che85 there is no particular issue with running normalization on separate keys. The main problem is the task order and the fact that we have to take care of many cases.

The issue I've faced with BRATS using the previous inference script is that the ConcatItemsd transform was not stacking the volumes in the correct order. There is no way to define that order using that transform.

Also, the previous transform loads the images and checks that all volume sizes match (https://github.com/lassoan/SlicerMONAIAuto3DSeg/blob/main/MONAIAuto3DSeg/Scripts/auto3dseg_segresnet_inference.py#L78-L85). This adds another level of complexity when having 4 volumes as BRATS - unless we load the volumes, check the sizes, resize if needed, save the volumes and then load again.

But I think Andras' idea of having a separate inference for BRATS makes more sense. I'll try to have that one later.

Thanks again for your comments.

[diazandr3s]

Inference script updated to work in all cases, including BRATS - single file

[diazandr3s]

Lot of code is moved around, so it is hard for me to track down if no functional changes are made to the existing code, but I take your word for it.

The array manipulations for brats are a bit cryptic, I don't see how convert 3 channel into 1 channel ints is implemented by those few lines, so it would be nice to add a little more explanation.

Thanks for checking this, @lassoan

Lot of code is moved around, so it is hard for me to track down if no functional changes are made to the existing code, but I take your word for it.

Most of the code is a repetition to manage BRATS and non-BRATS use cases.

The array manipulations for brats are a bit cryptic, I don't see how convert 3 channel into 1 channel ints is implemented by those few lines, so it would be nice to add a little more explanation.

BRATS model outputs 3 channels for the different overlapping tumour regions. I just merged them into a single channel. Let me add a comment to that.