search

SCIInstitute / ShapeWorks

ShapeWorks
http://sciinstitute.github.io/ShapeWorks/
Other
103 stars 32 forks source link

ICP Functionality in GroomUtils #653

Closed archanasri closed 4 years ago

archanasri commented 4 years ago

The transformation matrix created by ICP is not used in GroomUtils but ICP takes care of setting all the image sizes to that of the reference (target) image which is important for the next step (cropping) of the grooming process. For now, this is taken care of by the resize command but the underlying issue needs to be fixed.

archanasri commented 4 years ago

@sheryjoe any suggestions?

sheryjoe commented 4 years ago

What should resize do? Based on the name, it should only "resize", i.e, scale, the given image to a given logical size, or voxel spacing. Applying the transformation to an image is different. If ICP is setting the image size to the reference image, this means that ICP is already applying (via a resample and transformation filters) the transformation matrix. To keep modularity, this should translate to an icp command that only computes the transformation matrix and an "apply" or "resample" command that takes in the transformation matrix and apply it to the image.

I don't think that icp in groom utils is returning the transformation matrix for the purpose of applying it, rather keeping track of the transformation applied to the segmentations either to provide a way back to the original (given) coordinate system and/or apply the same transformation to the corresponding image data.

cchriste commented 4 years ago

That’s exactly what it does. The command line version, which calls the imo still far too specialized (I always say “idiosyncratic” with a negative implication, but we’re all special, right?) ImageUtils function. It wraps this functionality. The python wrapped api will not require this much and make the operation more transparent.

...which I’m the case of the past unmeasurable amount of time of using this executable has been exactly a resize and nothing else.

On Mon, Aug 3, 2020 at 6:06 PM Shireen Elhabian notifications@github.com wrote:

What should resize do? Based on the name, it should only "resize", i.e, scale, the given image to a given logical size, or voxel spacing. Applying the transformation to an image is different. If ICP is setting the image size to the reference image, this means that ICP is already applying (via a resample and transformation filters) the transformation matrix. To keep modularity, this should translate to an icp command that only computes the transformation matrix and an "apply" or "resample" command that takes in the transformation matrix and apply it to the image.

I don't think that icp in groom utils is returning the transformation matrix for the purpose of applying it, rather keeping track of the transformation applied to the segmentations either to provide a way back to the original (given) coordinate system and/or apply the same transformation to the corresponding image data.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/SCIInstitute/ShapeWorks/issues/653#issuecomment-668299398, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJT3EMP5ONC4GY2DICBJZDR65GIJANCNFSM4PT2OSLA .

cchriste commented 4 years ago

Let's conduct some tests w/ and w/o the icp alignment step w/ LA. 1a. ask whether the images seem more aligned or not 1b. compare final results and try to understand why using this transform makes them worse (we think it did)

2a. Try to manually modify the data to see if there's any benefit even when the data are more out of line (30 degrees) 2b. compare results

  1. Potential win overall: don't apply, just pass them along (depending on use case): These transforms could be supplied to the Procrustes optimizer rather than pre-applied to the images (thereby worsening the data quality). This could also include elimination of centering as part of the data grooming. The reason to keep this is there might be some reason to groom/align without successive optimization steps (e.g., deep learning directly on the images).
AtefehKashani commented 4 years ago
  1. in the icp alignment, we set the target image as reference, and all the input data should have the same size as the reference image, as @archanasri mentioned. It is critical for the next step in the pipeline(cropping) and optimization. 2 . That's correct! The transformation matrix is not used in the GroomUtils. We are writing these matrices to use them for down the road tasks. ( using this transformation matrix to align the MRI images or if we have a sequence of images for the same subject align all of them with this transformation matrix)
  2. @cchriste LA data is not a good example to test icp, since they are somehow pre-aligned, and you are right you can't see any huge difference after applying the icp. But I have tested this transformation for other data set like Right ventricle, which are not in good alignment, and it changed them in the desired way and helped the optimization a lot. I agree Procrustes can take care of miss alignment and improve optimization, but having the input data roughly aligned using icp helps the optimization, and it will converge in the less number of iterations.
archanasri commented 4 years ago

@AtefehKashani From what I understand, ICP is just a standalone command that is used outside of GroomUtils for down the road tasks for the current use cases. For use cases like the right ventricle, we would need ICP in GroomUtils. Is that right?

AtefehKashani commented 4 years ago

@archanasri ICP is a standalone command, but we need to keep it in the GroomUtils. It should be applied before cropping and optimization, please don't exclude it from GroomUtils. All I am saying is that, you may not notice its effect for some data sets like LA, but for other data set like VR it is essential.

archanasri commented 4 years ago

@AtefehKashani When the correct ICP is used for the ellipsoid use case, it produces bad results.

akenmorris commented 4 years ago

What's the precise cause of this? Is it due to the combination of the low resolution of the ellipsoid images and the resampling? Are we performing this step on binary segmentations or distance transforms? If the former, are we using a nearest neighbor interpolation or perfoming a binarize operation? Neither will be ideal.

archanasri commented 4 years ago

@akenmorris We are performing this on binary segmentations with a nearest neighbor interpolation.

akenmorris commented 4 years ago

I wouldn’t be surprised at the result then. I don’t think the ICP tool should be used in this way.

archanasri commented 4 years ago

@akenmorris What about using a linear interpolation?

akenmorris commented 4 years ago

I don't think that will make a difference in our current pipeline. Later on, I see an extractLabel(1.0) call that will effectively binarize the result.

cchriste commented 4 years ago

ICP works as advertised. We should use Image::resample instead of Image::resize in order to correctly resolve the LA fix in pull request #651 (details in #659 and #661 which will be resolved on the shapeworks_improv branch containing the LA fix pull).

The more general issue of not applying transforms until the last minute is now issue #660. It includes many other functions besides ICP.

I am therefore closing this issue.