Project-MONAI / monai-deploy-app-sdk

MONAI Deploy App SDK offers a framework and associated tools to design, develop and verify AI-driven applications in the healthcare imaging domain.
Apache License 2.0
94 stars 48 forks source link

[FEA] Ensemble MergeAndMask Operator #422

Closed jtetrea closed 1 year ago

jtetrea commented 1 year ago

Is your feature request related to a problem? Please describe.

There are some examples of MONAI MAPs with multiple models, however these are typically for different organs (e.g. liver and spleen). In some cases, a user may want to run an ensemble of AI models that were trained on different splits of a dataset. This can improve segmentation performance on things like small lesions that are already hard to label, train on, and validate as correct. In this case, there may be many segmentations of the same region that would need to be somehow filtered or combined in order to produce a single result.

Describe the solution you'd like

Suggest creating a "MergeAndMask" operator that provides a basic template of different post-processing operations that may be desirable after running ensemble inference with multiple AI models.

Describe alternatives you've considered

The alternative would be to add specialized post-processing and communication to the end of the individual inference operators, which imposes overhead. The InferenceOperator itself could be extended to take multiple models as input, but again this seems like a less straightforward option that would require additional configuration. With a single operator to handle any desired AI post-processing, many other research extensions and variations (e.g. dilation/erosion, creating specific probability maps) would all be contained in the same place for easy experimentation and iteration.

Additional context

Working on a current version of this that simply does logical AND of several masks, but want to contribute a more general template operator for this function.

jtetrea commented 1 year ago

Proposed operator added to branch JT/MergeVolume. New operator resamples and merges (currently 3) input images into a single multi-channel image.