Miscellaneous - Generated mask is shown with different origin from the original image

[diazandr3s]

Describe the bug Slicer shows the automatic label with a different origin. Apparently, it is related to the way ITK interprets the nifty header. From this discussion, it seems nibabel and nifty differ in the way they interpret the nifty header (https://github.com/nipy/nibabel/issues/863)

CC @wyli

To Reproduce Using the Fusion Challenge dataset MICCAI 2012 (OASIS project) to train highresnet for Brain parcellation.

Fusion Challenge dataset MICCAI 2012: 35 brain MRI scans obtained from the OASIS project. The manual brain segmentations of these images were produced by Neuromorphometrics, Inc. (http://Neuromorphometrics.com/) using the brainCOLOR labeling protocol.

Expected behavior To show the label in the same origin as the original image

Screenshots The workaround for this is to first change the affine of images and labels. Then use MONAI Label to train the model. See the affine changes in the following images

Original Affine:

Modified affine:

[lassoan]

Storing a 3D image does not have to be complicated, only NIFTI has these orientation issues (and a host of other problems). We lost so much time over the years with debugging various NIFTI problems and developing workarounds - this really has to stop. I would recommend to not use NIFTI format and discourage everyone else from using it. It would be hard to convince neuroimaging people to start using something else, but there is no reason for other communities to suffer.

If you have some original data in NIFTI format then you can convert it correctly (using all workarounds that are needed) to NRRD once, and then you won't have issues with inconsistencies related to image orientations, how to interpret the redundant sform/qform fields, what to do with non-orthogonal image axes, corrupted files, and all the other problems (see some of them here).

NRRD is certainly very limited but at least it is very simple. If something more sophisticated is needed then something like https://pypi.org/project/ome-zarr/ could be adopted.

[curtislisle]

I have also heard that ITK's NIFTI implementation has some troubles. I recommend allowing other image formats in MONAILabel. NRRD would be fine with me. I see a few places in the MONAILabel code where .nii and .nii.gz are the only allowed extensions. On the ZARR format, recent ITK releases have added support for it. It is space efficient and supports compression. If Zarr and NRRD were added as supported formats (ITK handles them), that would provide good alternatives. Thanks for this consideration.

[wyli]

I'm looking at this specific issue, I'm wondering if it is possible to set 3D Slicer to use the sform in nifti for visualisation? (looks like 3D slicer would only look at the qform, but I couldn't find the relevant docs.). That would be an easy way out for this ticket.

[wyli]

Another workaround in this context is to use tools to modify the original image such as fslorient -copyqform2sform 1000_3.nii to make the sform, qform consistent.

[curtislisle]

I see from other packages, that sform is preferred now over qform by the community (see reference from ANTs, a major registrtion package: https://github.com/ANTsX/ANTs/issues/1141 ) . I support the suggestion of making the header consistent through fslorient over modifying 3D Slicer. Slicer fixes need to be done slowly and carefully, as it is currently used in real-time surgical applications.

[lassoan]

I'm looking at this specific issue, I'm wondering if it is possible to set 3D Slicer to use the sform in nifti for visualisation? (looks like 3D slicer would only look at the qform, but I couldn't find the relevant docs.). That would be an easy way out for this ticket.

Slicer uses ITK to read images from NIFTI file format, so Slicer does what all other ITK-based applications do (that use the current ITK version).

I don't know what strategy ITK implements exactly now, but it does not really matter, because there is no common agreement in the neuroimaging community about how to use sform/qform in certain situations, so there is no single "correct" solution. To complicate matters further, the behavior changed many times over the past two decades - in different versions of NIFTI format and in different versions of its implementations.

[wyli]

Slicer uses ITK to read images from NIFTI file format, so Slicer does what all other ITK-based applications do (that use the current ITK version).

I guess that could be true in theory, but in practice that is not correct... ITK-snap loads the image+segmentation properly, with a warning message saying 'header mismatch and the (segmentation) header is ignored' -- while Slicer shows the mismatched image and seg. quietly.

[SachidanandAlle]

I have also heard that ITK's NIFTI implementation has some troubles. I recommend allowing other image formats in MONAILabel. NRRD would be fine with me. I see a few places in the MONAILabel code where .nii and .nii.gz are the only allowed extensions. On the ZARR format, recent ITK releases have added support for it. It is space efficient and supports compression. If Zarr and NRRD were added as supported formats (ITK handles them), that would provide good alternatives. Thanks for this consideration.

monailabel supports other extensions.. simple way to force output to be nrrd is... in your sample app (go to info.yaml) and add following item in config->infer section

config:
  infer:
    device: cuda
    result_extension: .nrrd

or you can send it per request in your infer API (in the params json) For loading any image, you should be using LoadImaged(keys="image") transform from MONAI

[diazandr3s]

LoadImaged transform in MONAI uses different readers (ITKReader(), NumpyReader(), PILReader(), NibabelReader()). Here is the list of formats ITKReader supports

[lassoan]

ITK-snap loads the image+segmentation properly, with a warning message saying 'header mismatch and the (segmentation) header is ignored' -- while Slicer shows the mismatched image and seg. quietly.

Latest ITK-Snap version was released more than 2 years ago. It is very far from latest ITK.

ITK-Snap complains about header mismatch because it just cannot handle images and segmentations with different geometries (origin, spacing, axis directions, or extents). It is completely valid to use images and segmentations with different geometries (for example, the segmentation may use isotropic spacing and/or smaller spacing than the input images), and rejecting such images/segmentations or ignoring the header information is a not a feature but a limitation of ITK-Snap.

[lassoan]

monailabel supports other extensions.. simple way to force output to be nrrd is... in your sample app (go to info.yaml) and add following item in config->infer section

LoadImaged transform in MONAI uses different readers (ITKReader(), NumpyReader(), PILReader(), NibabelReader()). Here is the list of formats ITKReader supports

Thanks for all the information. It is clear that MONAI supports multiple file formats. The problem is the NIFTI format is overrepresented in MONAI examples and sample data sets and seems to be used by default when a 3D image file format is needed. This may give the false impression to newcomers that it is OK to use NIFTI as a general-purpose 3D image file format.

[SachidanandAlle]

no more updates on this issue. closing now..

for e.g. monailabel server supports multiple output extensions (for e.g. nrrd is used for writing output masks to get consumed in ohif)

[diazandr3s]

Opening this issue again. I'm facing the same problem in Slicer when using Multi-Atlas Labeling Beyond the Cranial Vault dataset. Should we encourage researchers to convert NIFTI images into NRRD format when using MONAI Label? cc @wyli @finalelement

[diazandr3s]

Just converted the NIFTI images into NRRD format and it seems to work well

[lassoan]

Should we encourage researchers to convert NIFTI images into NRRD format when using MONAI Label?

I would strongly support this. Nifti is developed for functional MRI image interchange and has several serious issues that make it a bad choice as general-purpose medical image computing file format.

[mjorgecardoso]

NIFTI is not worse or better than NRRD. It's just more complicated. You can do certain things better with it, but you can also screw up the data more often if you don't know what you are doing.

Non-orthogonal axis issue can be stored in the sform, and non-ortogonal axis don't have a representation as tensors anyway - they would need to be resampled to a ortogonal grid; and the file corruption issues are just a limitation of how ITK reads data if there are multiple versions of the data.

The issue with the "Fusion Challenge dataset MICCAI 2012" is probably caused by a combination of the sform code 2 (the data is wrong) and the ITK reader doing something funky. Moving to NRRD would only make the problem a NIFTI-to-NRRD conversion issue than a MONAI issue.

We should definitely support more data formats, but I'm not sure what the argument being made for/against a data standard really is.

[SachidanandAlle]

Unfortunately MSD Datasets are in nifti format. It shall be great if someone can convert them to .nrrd and share for public download. Then all our examples can refer .nrrd as default.

[mjorgecardoso]

The MSD nifti data is correct, and does not use any SForm info. Thus, any nifti reader should do a good job.

[SachidanandAlle]

@diazandr3s can you verify if this is still an issue.. if not then we can close this issue