search

dbolya / yolact

A simple, fully convolutional model for real-time instance segmentation.
MIT License
5.04k stars 1.32k forks source link

Label & Mask #461

Open doit-ceo opened 4 years ago

doit-ceo commented 4 years ago

I have question regarding best practice for masks, the preferred mask for yolact. This is related to overlapping objects and mainly when they intersect, like the image below, would that confuse the network ? should I avoid training with such images ? this will happen in real live ! I tried to make segmentation into 2 separate polygons and LabelMe not allowing this !

When object is a standalone it's easy ...

seg-exm2

But what if we have something like this ? is the mask still correct ?

seg-example

Thanks in advance.

abhigoku10 commented 4 years ago

@doit-ceo this is interesting problem statement , i think we this will confuse the network most of the time . did u test it on small dataset ??

ic commented 4 years ago

The COCO format supports multiple polygon per object (MPPO). If your base model has been trained with COCO, this suggests the "under can" annotation would be a set of two polygons. Other formats may not provide this capability, or not the same way, so it may be worth confirming that first.

LabelMe is a generic tool that can export to COCO, but, as you found, it seems LabelMe does not support MPPO. So in the end there are two problems here, actually unrelated to Yolact...

COCO is affirmative on the first question. If the tool does not support, it is possible to annotate an image as many times as you need polygons for an object, export to COCO, and rewrite the related annotations into 1 MPPO. I can tell you the latter works, but it requires time-consuming care.

One alternative is to chose a tool that supports MPPO. There are many out there. Unfortunately I do not remember which (perhaps ImageTagger or CVAT ?).

As far as Yolact is concerned, same thinking and question as @abhigoku10 :-)