GlobalMutualInformationLoss with different bin distribution

[kate-sann5100]

Is your feature request related to a problem? Please describe. The current implementation of GlobalMutualInformationLoss can only apply Parzen windowing with bins distributed evenly between 0 and 1 instead of customise bins according to the input distribution. This lead to two problems: 1) The inputs (target and pred) must range between 0 and 1 2) It is hard to benchmark this loss against the implementation by other packages (e.g antspyx)

Describe the solution you'd like Reimplement GlobalMutualInformationLoss.

Describe alternatives you've considered N/A

Additional context N/A

[wyli]

Thanks, I've added this to the image registration milestone.

[kate-sann5100]

After searching, I have found the following mutual information implementations by established repositories:

1. Scikit-image(skimage.metrics.normalized_mutual_information)
2. NiftiReg implemented normalised mutual information
3. ANTsPY(ants.image_mutual_information) (source code)
4. ITK (itkJointHistogramMutualInformationImageToImageMetric itkNormalizedMutualInformationHistogramImageToImageMetric itkMattesMutualInformationImageToImageMetric)

The most similar implementation to ours is itkMattesMutualInformationImageToImageMetric, while ants.image_mutual_information is its python wrapper.

[kate-sann5100]

The main difference is that while Mattes mutual information adopts B-Spline kernel to calculate the weight each voxel adds to respective bins, our implementation(similar to DeepReg and VoxelMorph) adopts Gaussian probability.

[kate-sann5100]

The following steps are planned:

1. Implement B-Spline kernel as an option, together with gaussian prob (our original implementation).
2. Benchmark our implementation with B-Spline kernel with ants.image_mutual_information. This could be achieved either by getting the same result when using the same input or show changes in same directions as the input changes.
3. Benchmark our implementation with Gaussian probability against B-Spline kernel by adjusting the sigma parameter.

[kate-sann5100]

Current and future progress will be uploaded to this pull request https://github.com/Project-MONAI/MONAI/pull/2847

[kate-sann5100]

@wyli @YipengHu

[kate-sann5100]

• The B-Spline kernel has been implemented. (updated in pull request #2847, but unit test is to be written.)
• The following image visualises the MattesMutualInformation calculated on an 3D MRI image of male pelvic organs and itself after translation (implemented with monai.transforms.Affined(translate_params=(i, i, i))) at different value of translate param i by our and ANTsPY's implementation.
• Though value difference occurs between our and ANTsPY's implementation
• We observe a monotonic increase in loss as the translation param I increases
• Moreover, the difference between our and ANTsPY's implementation decreases as the loss increases

[kate-sann5100]

[YipengHu]

• although not expecting otherwise, maybe try the other two scenarios to be comprehensive:
  1. varying the three parameters independently with randomly sampled, fixed the other two
  2. varying all three randomly than plot the sorted Euclidean distance vs. MI.
• looking at the graph, it may be trivial, but provide a piece of stats for testing monotonic relationship, perhaps Spearman rank-order correlation? @kate-sann5100

[wyli]

looks cool, after the verifications, if we can reproduce the blue line in the unit test, it'll be a nice mile stone, what do you think?

[YipengHu]

looks cool, after the verifications, if we can reproduce the blue line in the unit test, it'll be a nice mile stone, what do you think?

Agreed.

[kate-sann5100]

I have also visualised the trending of MI against rotation

Moreover, here are the results under the two scenarios suggested by @YipengHu

1. varying only one parameter with randomly sampled

2. varying all three randomly then plot the sorted MSE vs. MI.

[wyli]

thanks, is this based on the latest implementation? I've updated the rounding option https://github.com/Project-MONAI/MONAI/pull/2847/commits/e5c6db061ab4b8c26f4209621fb0f99e1129df05 to make the b-spline option differentiable (see the test https://github.com/kate-sann5100/MONAI/blob/e5c6db061ab4b8c26f4209621fb0f99e1129df05/tests/test_reg_loss_integration.py#L39).

[kate-sann5100]

I have updated a better written result in my forked MONAI (to avoid adding new dependencies to this main repository)

The unmentioned results are achieved by the current code in pull request 3196, where a bug fix has been made.

I will add the benchmarking results into the unit-testing within this week in the same pull request if you are happy with it.

Advices are welcomed on how to modify the structure and writing of the benchmarking report.

@wyli @YipengHu

[wyli]

Thanks, it looks great, I put the testing data in the project share drive here: https://drive.google.com/uc?id=17tsDLvG_GZm7a4fCVMCv-KyDx0hqq1ji, you can follow this approach to set up the tests with a data downloading step: https://github.com/Project-MONAI/MONAI/blob/7dc364cdf5524fd6a2d24d2abc63894651ce4c61/tests/test_patch_wsi_dataset.py#L109-L110

The markdown documentation could go to the tutorial repo, perhaps the modules folder https://github.com/Project-MONAI/tutorials/tree/master/modules.

@kate-sann5100 @YipengHu I think we can merge this part and have a chat next week?

[YipengHu]

Fine with me, although the equations do not show up well on my dark theme.