Project-MONAI / MONAI

AI Toolkit for Healthcare Imaging
https://monai.io/
Apache License 2.0
5.67k stars 1.04k forks source link

Include ObjectID for create_test_image_2d #5267

Open JHancox opened 1 year ago

JHancox commented 1 year ago

Is your feature request related to a problem? Please describe. create_test_image_2d is great, but at the moment, you can get the class ID, but not the object instance ID. This means that two overlapping objects effectively become one large object (i.e. the boundary is undefined).

Describe the solution you'd like It would be really useful if the output could include a channel for each object ID. This would allow individual objects to be discerned

Describe alternatives you've considered For now, I have created my own class to accomplish this, but would prefer to use the Monai Core version

Additional context This is a useful feature for testing loss when it comes to nucleus instance segmentation in histopathology, where there are frequently overlapping nuclei.

mingxin-zheng commented 1 year ago

Hi @JHancox , thanks for bringing up the issue. I am sorry that I don't understand it very well. Currently the create_test_image_2d would output an image with the class_id, or has the set a dimension for the channel. Do you want to use the channel for the object instance ID instead of the class ID. If that's the case, maybe you can just put the number of instance in the num_seg_classess argument?

JHancox commented 1 year ago

Hi @JHancox , thanks for bringing up the issue. I am sorry that I don't understand it very well. Currently the create_test_image_2d would output an image with the class_id, or has the set a dimension for the channel. Do you want to use the channel for the object instance ID instead of the class ID. If that's the case, maybe you can just put the number of instance in the num_seg_classess argument?

Thanks @mingxin-zheng. The problem that I am trying to resolve is that the current feature will only output a per-pixel classID. This means that if I have specified, say, 10 objects with 2 classes then I will get per pixel values ranging from 0-2. However, if any of the objects overlap then they essentially become one object, with no way of telling which pixel belongs to which original object. To get around this would require a separate channel in which each pixel also has an objectID ranging from 0-10. Happy to submit a PR for this, since I already created an extension to cater for it.

mingxin-zheng commented 1 year ago

Thanks @JHancox for the reply. How would you organize the output arrays to include (1) class_id (2) instance_id at the same time?

JHancox commented 1 year ago

I guess that's up for discussion, but to maintain compatibility, another channel could be added to the labels ndarray if an optional 'return_object_id' parameter is set to True.

mingxin-zheng commented 1 year ago

Adding a dimension could be risky because there are lots of unit-tests dependency. What do you think @wyli ?

Inspired by something like this, maybe we can slightly modified the the current class_id, for example, with a user argument to enable the encoding/decoding the number from/to class_ID and instance_ID?

wyli commented 1 year ago

we can add a return_instance_id: bool = False argument to https://github.com/Project-MONAI/MONAI/blob/da1c32abbd77f9a8c2674f764be78aeb4828c1b6/monai/data/synthetic.py#L21 (and the 3d version). When it's set to True, the function return noisyimage, instance_ids, False to return return noisyimage, labels (current default). what do you think?

JHancox commented 1 year ago

The problem is that we really need the class ids too (greedy I know). Could we return a tuple of (class_id, instance_id) when that parameter is set? (Although, I think I prefer the extra dimension for this case because otherwise you need to unpack the tuple for each pixel.)

wyli commented 1 year ago

sure returning three elements is also fine:

if return_instance_id:
    return noisyimage, labels, instance_ids
return noisyimage, labels