Grabcut Implementation

[charliebudd]

Grabcut (presented here) is a method of segmenting a reigion based on a sparse initial labeling.

A color model is employed (GMM is common) to determine color distibutions of the labeled reigions. These distributions are then used to asign proberbilities to unlabled regions of the image. Graphcut is then employed to optimize the boundries of the reigions based on the morphological boundries of the original image. This process may be iterated on until convergence.

This will be useful in MONAI for both interactive segmentation and as a post-processing step for segmentation algorithms.

I see this being split into three PRs. GMM, Graphcut, and finally Grabcut. Each would provide useful functionality induvidually.

[tvercaut]

For the record, a good starting point in terms of implementation is the one from @luiscarlosgph https://github.com/luiscarlosgph/grabcut Itself mostly borrowing from that in OpenCV + code from old CUDA samples

[tvercaut]

A few more components are now available, with some starting to be used in MONAILabel.

@masadcv implemented a wrapper for OpenCV's graphcut algorithm here: https://github.com/masadcv/numpymaxflow There is also a wrapper for pytorch directly but it faces some issues related to packaging (https://github.com/masadcv/torchmaxflow). Ideally, this would be integrated in MONAI proper and the packaging issue addressed in MONAI.

It also seems that the JIT-based compilation used for the GMM code here may not solve all packaging issues, see: https://github.com/Project-MONAI/MONAILabel/pull/763#discussion_r862096992 Until this is solved, would it be possible to have a pure-python fallback for the GMM? An implementation for it can for example be borrowed from https://github.com/ldeecke/gmm-torch

[masadcv]

I may be able to help integrate torchmaxflow into MONAI (if this is something of interest here). As @tvercaut pointed out, this may first need packaging to be addressed such that C++ based torch implementations can be packaged and deployed within MONAI.

We have already seen significance of using maxflow algorithm in MONAI Label (https://github.com/Project-MONAI/MONAILabel/pull/731) and I am sure it will be useful for other segmentation processing applications as well.

[tvercaut]

The packaging question and c++ implementation of graphcut can be dealt with independently I think. We can already get a JIT-based approach in MONAI and once a more robust packaging approach is ready, it can be applied to all c++ extensions already available. The packaging will anyway require cleanup as there are already 2 different approaches to includin c++ extensions in MONAI: https://github.com/Project-MONAI/MONAI/tree/dev/monai/_extensions https://github.com/Project-MONAI/MONAI/tree/dev/monai/csrc

[masadcv]

Agreed, this can be addressed independently from packaging. I can add a JIT module for this.

In terms of packaging, it is certainly needed in order to use such MONAI functions as dependency in other projects—we have already seen a number of issues with dependencies that required compilation (e.g. SImpleCRF dependency issues in MONAI Label see for example: https://github.com/Project-MONAI/MONAILabel/issues/605 and https://github.com/Project-MONAI/MONAILabel/issues/719) and hence it would be good to address this for MONAI C++ functions.

[tvercaut]

@masadcv: Unless there is an existing MONAI-related issue on packaging, can you create one?