Difficulties with unigradicon-warp

russchua commented 3 months ago

Hi, thanks for making uniGradICON.

The instructions look simple enough and unigradicon-register works fine. However I'm having trouble with unigradicon-warp when trying to warp labels.

This works fine: unigradicon-register --fixed=$fixed_image_file --fixed_modality=mri --moving=$moving_image_file --moving_modality=mri --transform_out=$transform_path --warped_moving_out=$warped_image_path --io_iterations None

This has issues: unigradicon-warp --fixed $fixed_image_file --moving $moving_label_file --transform $transform_path --warped_moving_out $warped_label_path --nearest_neighbor

Issues: File "/unigradicon_testing_space/github_pip/uniGradICON/src/unigradicon/init.py", line 312, in wa rp_command interpolator = itk.NearestNeighborInterpolateImageFunction.New(moving) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/unigradicon_conda/lib/python3.11/site-packages/itk/support/template_class.py", line 734, in New raise itk.TemplateTypeError(self, input_type) itk.support.extras.TemplateTypeError: itk.NearestNeighborInterpolateImageFunction is not wrapped for input type None.

To limit the size of the package, only a limited number of types are available in ITK Python. To print the supported types, run the following command in your python environment:

itk.NearestNeighborInterpolateImageFunction.GetTypes()

Possible solutions:

If you are an application user: Convert your input image into a supported format (see below). Contact developer to report the issue.
If you are an application developer, force input images to be loaded in a supported pixel type.

e.g.: instance = itk.NearestNeighborInterpolateImageFunction[itk.Image[itk.SS,2], itk.D].New(my_input)
(Advanced) If you are an application developer, build ITK Python yourself and turned to ON the corresponding CMake option to wrap the pixel type or image dimension you need. When configuring ITK with CMake, you can set ITK_WRAP_${type} (replace ${type} with appropriate pixel type such as double). If you need to support images with 4 or 5 dimensions, you can add these dimensions to the list of dimensions in the CMake variable ITK_WRAP_IMAGE_DIMS.

Supported input types:

itk.Image[itk.SS,2] itk.Image[itk.UC,2] itk.Image[itk.US,2] itk.Image[itk.F,2] itk.Image[itk.D,2] itk.Image[itk.Vector[itk.F,2],2] itk.Image[itk.CovariantVector[itk.F,2],2] itk.Image[itk.RGBPixel[itk.UC],2] itk.Image[itk.RGBAPixel[itk.UC],2] itk.Image[itk.SS,3] itk.Image[itk.UC,3] itk.Image[itk.US,3] itk.Image[itk.F,3] itk.Image[itk.D,3] itk.Image[itk.Vector[itk.F,3],3] itk.Image[itk.CovariantVector[itk.F,3],3] itk.Image[itk.RGBPixel[itk.UC],3] itk.Image[itk.RGBAPixel[itk.UC],3] itk.Image[itk.SS,4] itk.Image[itk.UC,4] itk.Image[itk.US,4] itk.Image[itk.F,4] itk.Image[itk.D,4] itk.Image[itk.Vector[itk.F,4],4] itk.Image[itk.CovariantVector[itk.F,4],4] itk.Image[itk.RGBPixel[itk.UC],4] itk.Image[itk.RGBAPixel[itk.UC],4] itk.PhasedArray3DSpecialCoordinatesImage[itk.F] itk.PhasedArray3DSpecialCoordinatesImage[itk.UC]

HastingsGreer commented 3 months ago

Thank you for the detailed report! We will work on this

HastingsGreer commented 3 months ago

Hi! I have added a test to verify that unigradicon-warp works as expected on labelmaps, see https://github.com/uncbiag/uniGradICON/pull/20 . I haven't been able to replicate your issue on a generic label map, so I need more information on the labelmap that is causing the code to break for you.

Could you post the output of this code on your labelmap?

python

import itk
label = itk.imread("your-labelmap-here.nrrd")
print(label)
print(label.GetLargestPossibleRegion().GetSize())

russchua commented 3 months ago

Thank you, I am actually working with a NIFTI file. Here it is.

Image (0x8288290)
  RTTI typeinfo:   itk::Image<int, 3u>
  Reference Count: 1
  Modified Time: 786
  Debug: Off
  Object Name: 
  Observers: 
    none
  Source: (none)
  Source output name: (none)
  Release Data: Off
  Data Released: False
  Global Release Data: Off
  PipelineMTime: 604
  UpdateMTime: 785
  RealTimeStamp: 0 seconds 
  LargestPossibleRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [160, 160, 192]
  BufferedRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [160, 160, 192]
  RequestedRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [160, 160, 192]
  Spacing: [1, 1, 1]
  Origin: [0, 0, 0]
  Direction: 
1 0 0
0 1 0
0 0 1

  IndexToPointMatrix: 
1 0 0
0 1 0
0 0 1

  PointToIndexMatrix: 
1 0 0
0 1 0
0 0 1

  Inverse Direction: 
1 0 0
0 1 0
0 0 1

  PixelContainer: 
    ImportImageContainer (0x8288520)
      RTTI typeinfo:   itk::ImportImageContainer<unsigned long, int>
      Reference Count: 1
      Modified Time: 782
      Debug: Off
      Object Name: 
      Observers: 
        none
      Pointer: 0x8497cd0
      Container manages memory: true
      Size: 4915200
      Capacity: 4915200

itkSize3 ([160, 160, 192])

HastingsGreer commented 3 months ago

Ok! The test now fails, matching your report. It looks like itk doesn't support warping signed int labelmaps, which is surprising but knowing ITK, not that surprising haha. A first pass at a solution is to add a flag for casting int labelmaps to a type that the InterpolateImageFilters support, such as float or double- perhaps with a cast back before saving the image- I will look in to this

HastingsGreer commented 3 months ago

Hi! Your issues should now be fixed in the linked pull request. Until we merge it to main and update the pypi package, you can use it locally with

pip install git+https://github.com/uncbiag/unigradicon.git@test-unigradicon-warp-issue

russchua commented 3 months ago

When I do this, the label mask is not seen.

Is this likely because of the meta data contained in an image not otherwise captured in the label data? If so, how can I rectify this?

A print of the fixed_image, moving mask and the fixed_label is shown below

fixed_image:

Image (0xce6f340)
  RTTI typeinfo:   itk::Image<unsigned char, 3u>
  Reference Count: 1
  Modified Time: 3034
  Debug: Off
  Object Name: 
  Observers: 
    none
  Source: (none)
  Source output name: (none)
  Release Data: Off
  Data Released: False
  Global Release Data: Off
  PipelineMTime: 2850
  UpdateMTime: 3033
  RealTimeStamp: 0 seconds 
  LargestPossibleRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [160, 160, 192]
  BufferedRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [160, 160, 192]
  RequestedRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [160, 160, 192]
  Spacing: [1, 1, 1]
  Origin: [-77.6102, 71.9206, 72.2747]
  Direction: 
1 -1.11759e-08 8.46921e-09
-1.03464e-08 2.32831e-10 -1
1.11759e-08 -1 4.65661e-10

  IndexToPointMatrix: 
1 -1.11759e-08 8.46921e-09
-1.03464e-08 2.32831e-10 -1
1.11759e-08 -1 4.65661e-10

  PointToIndexMatrix: 
1 8.46921e-09 -1.11759e-08
1.11759e-08 -4.65661e-10 -1
-1.03464e-08 -1 -2.32831e-10

  Inverse Direction: 
1 8.46921e-09 -1.11759e-08
1.11759e-08 -4.65661e-10 -1
-1.03464e-08 -1 -2.32831e-10

  PixelContainer: 
    ImportImageContainer (0xce69460)
      RTTI typeinfo:   itk::ImportImageContainer<unsigned long, unsigned char>
      Reference Count: 1
      Modified Time: 3030
      Debug: Off
      Object Name: 
      Observers: 
        none
      Pointer: 0xd717d70
      Container manages memory: true
      Size: 4915200
      Capacity: 4915200

moving_label:

Image (0xd16fae0)
  RTTI typeinfo:   itk::Image<int, 3u>
  Reference Count: 1
  Modified Time: 3782
  Debug: Off
  Object Name: 
  Observers: 
    none
  Source: (none)
  Source output name: (none)
  Release Data: Off
  Data Released: False
  Global Release Data: Off
  PipelineMTime: 3600
  UpdateMTime: 3781
  RealTimeStamp: 0 seconds 
  LargestPossibleRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [160, 160, 192]
  BufferedRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [160, 160, 192]
  RequestedRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [160, 160, 192]
  Spacing: [1, 1, 1]
  Origin: [0, 0, 0]
  Direction: 
1 0 0
0 1 0
0 0 1

  IndexToPointMatrix: 
1 0 0
0 1 0
0 0 1

  PointToIndexMatrix: 
1 0 0
0 1 0
0 0 1

  Inverse Direction: 
1 0 0
0 1 0
0 0 1

  PixelContainer: 
    ImportImageContainer (0xd16c8e0)
      RTTI typeinfo:   itk::ImportImageContainer<unsigned long, int>
      Reference Count: 1
      Modified Time: 3778
      Debug: Off
      Object Name: 
      Observers: 
        none
      Pointer: 0xe077d90
      Container manages memory: true
      Size: 4915200
      Capacity: 4915200

fixed_label (as shown before):

Image (0xd05c300)
  RTTI typeinfo:   itk::Image<int, 3u>
  Reference Count: 1
  Modified Time: 4155
  Debug: Off
  Object Name: 
  Observers: 
    none
  Source: (none)
  Source output name: (none)
  Release Data: Off
  Data Released: False
  Global Release Data: Off
  PipelineMTime: 3973
  UpdateMTime: 4154
  RealTimeStamp: 0 seconds 
  LargestPossibleRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [160, 160, 192]
  BufferedRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [160, 160, 192]
  RequestedRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [160, 160, 192]
  Spacing: [1, 1, 1]
  Origin: [0, 0, 0]
  Direction: 
1 0 0
0 1 0
0 0 1

  IndexToPointMatrix: 
1 0 0
0 1 0
0 0 1

  PointToIndexMatrix: 
1 0 0
0 1 0
0 0 1

  Inverse Direction: 
1 0 0
0 1 0
0 0 1

  PixelContainer: 
    ImportImageContainer (0xd05c590)
      RTTI typeinfo:   itk::ImportImageContainer<unsigned long, int>
      Reference Count: 1
      Modified Time: 4151
      Debug: Off
      Object Name: 
      Observers: 
        none
      Pointer: 0xf337da0
      Container manages memory: true
      Size: 4915200
      Capacity: 4915200

HastingsGreer commented 3 months ago

Yes, most likely the spacing, origin and direction of the segmentation mask should match the image- since they appear to have the same Size. Ideally this could be fixed in whatever software produced the mask, as whenever images get separated from their spatial metadata, bugs lurk. However, in this case just copying the image metadata onto the mask should work- just verify that they align after you do this operation, by for example opening both images in 3D Slicer which is a metadata-aware image viewer. to copy the metadata, something like

In [8]: mask = itk.imread('mask'), img = itk.imread('img')

In [9]: mask.SetOrigin(img.GetOrigin())

In [10]: mask.SetDirection(img.GetDirection())

In [11]: mask.SetSpacing(img.GetSpacing())

In [12]: itk.imwrite(mask, "mask_with_metadata.nrrd")

uncbiag / uniGradICON

Difficulties with unigradicon-warp #18