Error loading supported file formats with `volume_loader`

tbirdso commented 5 months ago

Overview

A user reported an error in which they were not able to load data with the volume_loader operator.

Investigation

Verify image loading with each type of sample data
- NRRD
- MetaImage
- NIFTI
Investigate and address any observed loading issues

EDIT: Updated to reflect we are not sure what file type was failing. The error occurred while trying to set up the volume_rendering_xr application with custom data.

rbridgewaterML commented 5 months ago

Noting: I received an unclear message, File is not a supported volume format MRI_headerless.nrrd after using a headerless NRRD (As apparently NRRDs with attached headers are known to have issues, which might be good to document as well)

AndreasHeumann commented 5 months ago

The loader in currently supports NRRD data with detached header (https://teem.sourceforge.net/nrrd/format.html#detached) only. I agree that this is not documented clearly. There are no know issues with the MetaImage and NIFTI loaders.

tbirdso commented 5 months ago

Results from my local testing:

Unexpected Behavior

Compressed .nhdr volume fails to load with error: "NRRD unexpected value for encoding: gzip"
Uncompressed .nhdr volume loads, but rendering appears incorrect.

Verified Behavior

.mhd + .raw loads without error
.nii.gz loads without error
.mha load error: "File is not a supported volume format"
nrrd load error: "File is not a supported volume format"

Test Procedure

I tested in the volume_rendering app (which makes use of VolumeLoaderOp and VolumeRenderOp) as follows:

Verify default app build/run behavior with ./dev_container build_and_run volume_rendering
Launch the HoloHub container:
```
./dev_container launch
```

Use the ITK Python package to convert .mhd sample data to other supported formats under test:

pip install itk
pushd data/volume_rendering
python -c "import itk; image = itk.imread('highResCT.mha'); itk.imwrite(image, 'highResCT.<output-format>, compression=<True/False>)"
popd

Try running the volume_rendering app with the density image in the given file type:

/workspace/holohub$ /workspace/holohub/build/applications/volume_rendering/volume_rendering  --mask /workspace/holohub/data/volume_rendering/smoothmasks.seg.mhd --config /workspace/holohub/data/volume_rendering/config.json --density /workspace/holohub/data/volume_rendering/highResCT.<file-format>

Followup Notes

I see that we may have used uncompressed, detached-header NRRD files elsewhere (see bonsai). It looks like we have an implicit NRRD requirement that data is uncompressed. @AndreasHeumann do you know whether there may be additional constraints or underlying assumptions for NRRD volume loading, such as maybe expecting a uint8 type on load?
My recommendation is that we continue to direct users towards detached-header .mhd and compressed .nii.gz volumes for now. It is OK to not support all possible medical volume types in VolumeLoaderOp so long as external conversion tools are available.

AndreasHeumann commented 5 months ago

I see that we may have used uncompressed, detached-header NRRD files elsewhere (see [bonsai](https://klacansky.com/open- scivis-datasets/)). It looks like we have an implicit NRRD requirement that data is uncompressed. @AndreasHeumann do you know whether there may be additional constraints or underlying assumptions for NRRD volume loading, such as maybe expecting a uint8 type on load?

Uncompressed data is supported, but the loader is only checking for gz:

https://github.com/nvidia-holoscan/holohub/blob/main/operators/volume_loader/nrrd_loader.cpp#L81

Should be a simple fix to also add gzip which is a valid value as mentioned in the spec: https://teem.sourceforge.net/nrrd/format.html#encoding

All types except 64 bit types ([u]int64 and float64) are supported: https://teem.sourceforge.net/nrrd/format.html#type.

There are other limitations

txt, hex and bz2 encodings are not supported
big endian is not supported
linespkip and byteskip is not supported

nvidia-holoscan / holohub