Instance Segmentation Loss

[tonyyang1995]

Thanks for sharing the codes. The results are very impressive. I am a little confused about how instance segmentation works, and would you mind give me some suggestions like how to implement it. Many thanks.

[anthonyhu]

For instance segmentation of vehicles we create the following labels:

• segmentation mask of the vehicles
• centerness: likelihood of a pixel being the center of an instance (modelled as 2D Gaussian)
• offset: vector pointing to the closest instance center.

During training, we then have a loss for each of those labels.

When doing inference, we use the centerness to get instance centers, and combine the segmentation mask with the offset to assign remaining pixels to a particular instance. There are more details in the paper https://arxiv.org/abs/2104.10490 (Figure 3 and Appendix B.3)