search

Project-MONAI / MONAILabel

MONAI Label is an intelligent open source image labeling and learning tool.
https://docs.monai.io/projects/label
Apache License 2.0
591 stars 189 forks source link

Scribbles - Don't depend on logits (previous segmentation model) #265

Closed SachidanandAlle closed 3 years ago

SachidanandAlle commented 3 years ago

It shall be great if scribbles work independently of previous model output. Currently it depends on logits collected through previous segmentation action.

It will be ideal if we can make it work based on

masadcv commented 3 years ago

The simplest approach which could enable this comprises of:

  1. Building a likelihood model from foreground/background scribbles (typically in the form of foreground/background histograms)
  2. For each voxel in image, find out the corresponding likelihood for foreground/background label
  3. Normalise the resulting likelihood volume
  4. Take argmax of likelihood volume to get the assigned labels

An additional step can be there to regularise the resulting label volume by passing it through a standard CRF optimisation. ​

I see steps 1, 2, and 3 as an additional transform that gets activated when logits is not found.

masadcv commented 3 years ago

@SachidanandAlle this seems like we will have a separate application (e.g. segmentation_scribbles_coldstart or segmentation_scribbles_modelfree) to demonstrate/implement this functionality. Correct me if you think there is another way..

If we do end up implementing a new app, we need to think about putting scribbles transforms to some common place first (e.g. MONAI Core or MONAILabel Core) as this will be reusing the same code for optimising.

Please let me know your thoughts about this. I will pick this up early next week.

SachidanandAlle commented 3 years ago

You can start adding/moving all methods/algo under new package... you can define them as transforms or you can make them callable methods (so that they can be hooked has infer with/without model)

say all that re-useable code can go at https://github.com/Project-MONAI/MONAILabel/tree/main/monailabel/<scribbles> And this code shall be kind of library for developers to use/include them in their apps/notebooks.

later we can move this to monai-core for general purpose use-cases

On the other question related to independent scribbles.. You can start with an app called scribbles_xxx... if it's not organ specific then you can go with generic_scribbles and demonstrate all possible methods over there..

But please work on different pr once you have merged the existing one.. that way you can checkpoint current changes and then start working on updating/fixing new ones..