What does it take to add instance segmentation support?

facebookresearch / detr

End-to-End Object Detection with Transformers

Apache License 2.0

13.37k stars 2.41k forks source link

What does it take to add instance segmentation support? #79

Closed jackkwok closed 4 years ago

jackkwok commented 4 years ago

What does it take to add instance segmentation support? Any guidance appreciated.

alcinos commented 4 years ago

Hi @jackkwok Instance segmentation is already mostly there, since it's a subset of panoptic segmentation. What do you need exactly?

jackkwok commented 4 years ago

@alcinos : Unlike instance segmentation, each pixel in panoptic segmentation has only one label corresponding to instance i.e. there are no overlapping instances. I need to support overlapping instances.

alcinos commented 4 years ago

The non-overlapping property is enforced only in the panoptic post-processing (that you can entirely avoid if you don't want it), not in the loss at all. If you provide overlapping masks in your training, DETR will learn to output overlapping masks.

jackkwok commented 4 years ago

@alcinos: that's great because it's exactly what I need. Thank you for clarifying.