search

cvat-ai / cvat

Annotate better with CVAT, the industry-leading data engine for machine learning. Used and trusted by teams at any scale, for data of any scale.
https://cvat.ai
MIT License
12.2k stars 2.95k forks source link

How to create my own panoptic segmentation dataset? #4408

Closed Zjay closed 2 years ago

Zjay commented 2 years ago

Does CVAT support user creating panoptic labels like coco panoptic labels? Can I label stuff and things in the same image respectively?

zhiltsov-max commented 2 years ago

Hi! CVAT doesn't allow exporting into COCO segmentation formats directly, however it is possible with Datumaro. To do this, you need to:

  1. Annotate a task with polygons
  2. Export annotations in Datumaro format
  3. Unzip
  4. Run datum transform -t polygons_to_masks --overwrite <path/to/unzipped/dir>:datumaro
  5. Run datum convert -if datumaro -i <path/to/unzipped/dir> -o <output/dir/> -f coco -- --save-images
Zjay commented 2 years ago

Hi! CVAT doesn't allow exporting into COCO segmentation formats directly, however it is possible with Datumaro. To do this, you need to:

  1. Annotate a task with polygons
  2. Export annotations in Datumaro format
  3. Unzip
  4. Run datum transform -t polygons_to_masks --overwrite <path/to/unzipped/dir>:datumaro
  5. Run datum convert -if datumaro -i <path/to/unzipped/dir> -o <output/dir/> -f coco -- --save-images

Thanks for you reply! But I still have some issues about how to make panoptic segmentation labels.For instance, If an instance is divided into non-contiguous parts, how should I label them? And how to guarantee that there are no unlabeled areas in the image？ I'm not confused about how to export the dataset, but how to use CVAT make panoptic segmentation labels.

arronmabrey commented 2 years ago

I'm also wondering about non-contiguous parts, I've been trying to read through the docs but see no mention of it. Also the statement using polygons, does this exclude the use of the new ellipses style making in cvat 2.0

nmanovic commented 2 years ago

@arronmabrey , @Zjay , there are two ways to annotate non-contiguous parts:

  1. Annotate them as separate objects and group them after that: https://openvinotoolkit.github.io/cvat/docs/manual/advanced/shape-grouping/
  2. Annotate non-contiguous pats as one object. Draw above the object a new shape which is in foreground: https://openvinotoolkit.github.io/cvat/docs/manual/advanced/annotation-with-polygons/creating-mask/
Zjay commented 2 years ago

@arronmabrey , @Zjay , there are two ways to annotate non-contiguous parts:

  1. Annotate them as separate objects and group them after that: https://openvinotoolkit.github.io/cvat/docs/manual/advanced/shape-grouping/
  2. Annotate non-contiguous pats as one object. Draw above the object a new shape which is in foreground: https://openvinotoolkit.github.io/cvat/docs/manual/advanced/annotation-with-polygons/creating-mask/

Thanks a lot! I still have some questions about the doc. In the second link: image image Notice that it is currently impossible to have a single instance number for internal shapes (they will be merged into the largest one and then covered by “holes”)

Does it mean this method just work for semantic segmentation? In instance mode, it will lose efficacy. Is there any possible way for me to annotate an instance label that is surrounded by another instance label?

nmanovic commented 2 years ago

@Zjay , probably it is better just try on CVAT.org. At the moment I don't understand which features are missing for your case. CVAT was used by many individuals and companies to annotate data for panoptic segmentation. If you are missing a feature, just describe it.

If you export results as a mask in CVAT, the mask should have the same color for the same instance even it is split on multiple parts.

Zjay commented 2 years ago

@Zjay , probably it is better just try on CVAT.org. At the moment I don't understand which features are missing for your case. CVAT was used by many individuals and companies to annotate data for panoptic segmentation. If you are missing a feature, just describe it.

If you export results as a mask in CVAT, the mask should have the same color for the same instance even it is split on multiple parts.

Thanks for your suggestions and patience. I've tried several times. Because my team is looking for tools to create panoptic labels, but there is no explanation for that in the documentation. At this time I may not be able to meet all the problems I may encounter in the future. However, you mentioned that many teams are using your tools to complete panoptic label task. we can now use your tools with confidence.Thanks again！

rpl-alan-shangguan commented 1 year ago

Is there a plan to develop a function to export the coco panoptic format dataset?