search

Project-MONAI / MONAILabel

MONAI Label is an intelligent open source image labeling and learning tool.
https://docs.monai.io/projects/label
Apache License 2.0
597 stars 194 forks source link

Wrong Orientation/Origin for multichannel volumes (4D) using Slicer - Issue visualizing BRATS images on 3DSlicer #22

Closed SachidanandAlle closed 1 year ago

SachidanandAlle commented 3 years ago

BRATS and prostate, which have more than one modality. It seems we're having the same problem I had before. Once Slicer loaded the volume, I didn't find a way to get back all the modalities

This was something I left in the to-solve list. As workaround solution, I get the modalities before I loaded in Slicer

In addition to the multimodality issue in Slicer, the BRATS data seems to have a problem with the orientation

fepegar commented 3 years ago

This is because Slicer (which uses SimpleITK) cannot read these images and just chooses to assign some default orientation when images are 4D. A way to solve this is to first check the shape of the images (without Slicer) and use a custom loader if we know that Slicer will struggle reading the file.

diazandr3s commented 3 years ago

Another way to follow is showing ONLY one modality in Slicer, so the user provides points for DeepGrow, and visualizes the automatic label. However, for inference, we use all the available modalities.

fepegar commented 3 years ago

Even if you show the user only one modality, the image needs to be loaded correctly first. This might be relevant, although I think it's designed for images where time is 4th dimension: https://www.slicer.org/wiki/Documentation/Nightly/Modules/MultiVolumeExplorer

SachidanandAlle commented 3 years ago

Moving to backlog.. not a priority for 0.2 version release

diazandr3s commented 2 years ago

Currently, MONAI label allows the training of models on multichannel datasets (i.e. BRATS). However, visualizing 4D images in 3D Slicer is not supported.

Here are two examples of both image and prediction.

example2 example1

One shortcut for this is to discard orientation when loading the volume:

Instead of this:

slicer.util.loadVolume(image_file)

https://github.com/Project-MONAI/MONAILabel/blob/main/plugins/slicer/MONAILabel/MONAILabel.py#L1198

Do this:

slicer.util.loadVolume(image_file, {'discardOrientation': True})

example-3

Not sure about the effects of this on the other models.

CC @lassoan

lassoan commented 2 years ago

However, visualizing 4D images in 3D Slicer is not supported.

We can visualize 4D volumes, but loading of 4D Nifti files is currently not supported. You can load 4D NRRDs. We could add loading of 4D Nifiti images, which should not be a lot of work if ITK already supports it.

diazandr3s commented 2 years ago

image.nii.gz

qform code is set to NIFTI_XFORM_UNKNOWN. sform code is set to NIFTI_XFORM_ALIGNED_ANAT.

According to the specification 1: This method is not recommended, and is present mainly for compatibility with ANALYZE 7.5 files.

It might not be recommended because its interpretation is ambiguous.

Slicer relies on ITK to read this image and it ends up being interpreted as the image defined in the LPS coordinate system, that’s why the IJKtoRAS matrix is diag(-1,-1,1,1). Maybe this is not correct, but I’m not sure if there is anybody who can tell or if there is consensus on how to interpret this image.

label.nii.gz

Both qform and sform code are set to NIFTI_XFORM_SCANNER_ANAT.

According to the specification, The sform code should be set to either NIFTI_XFORM_UNKNOWN, NIFTI_XFORM_ALIGNED_ANAT, NIFTI_XFORM_TALAIRACH or NIFTI_XFORM_MNI_152.. So having NIFTI_XFORM_SCANNER_ANAT as sform code may make the file invalid.

Slicer relies on ITK to read this image and it is interpreted as the image defined in the RAS coordinate system, that’s why the IJKtoRAS matrix is diag(1,1,1,1). This is most likely correct, as NIfti usually uses RAS.

In summary both files may contain unsupported/invalid coordinate system definition.

You may get information on the history of this data set and advice on how to interpret on ITK or other neuroimaging forums. However, there is a practically unlimited number of ways how Nifti files can be messed up, so even if you figure out a solution that works for this particular data set, you’ll still run into problems with other data sets.

What you could do is to copy the metadata from the input file to the output file more closely. It is not enough to just specify the same origin, spacing, and axis directions, but copy the all the sform and qform related fields. Even if the input data set was invalid and/or ambiguous, the output will be wrong the same way.

Many thanks, @lassoan

SachidanandAlle commented 1 year ago

Not sure if we need any fix here. Closing the issue. As suggested, please start using nrrd whenever it's possible. MONAI and MONAILabel image loaders already support the same.

Also settings is available in Slicer Plugin to generate nrrd labels...

Otherwise 4D image depends on how UI supports/passes and Image Loader consumes. MONAILabel should not be any blocker here.