Masks with holes are imported with extra annotations in the Segmentation mask 1.1 format

[bisnow33]

My actions before raising this issue

• [x] Read/searched the docs
• [x] Searched past issues

I try to upload homemade annotation in cvat from Segmentation mask 1.1 format.

Expected Behaviour

Upload annotations.

Current Behaviour

I get the error on the screen shot captured (from the browser console):

The colors (243,243,243) mentioned is a color used for a label of SegmentedObjet . I checked in the labelmap.txt (from an the cvatExport.zip see below) but there is only SegmentationClass label information in the file and no mention of SegmentedObjet label.

Steps to Reproduce (for bugs)

To reproduce the error you can get the following zip file https://www.swisstransfer.com/d/45025776-3a00-441d-a6a1-c2b3ceac588d. The archive contain the following files :

rawData : a folder containing raw images

cvatExport.zip : manual annotations I made on cvat and exported in segmentation mask 1.1 format (to get a structure format example)

cvatImport.zip : annotation from homemade script and transformed as segmentation mask 1.1 format

Then to reproduce the error you can: 1- Create a task with image from rawData folder. In the task you have to create 1 label named: vegetation1 color code: 144,179,21 2 - Open the job 3 - Menu -> Upload annotations -> cvatImport.zip

Context

Your Environment

• Docker version: 20.10.19

• Operating System and version Linux

Best

[bsekachev]

@zhiltsov-max Could you please comment on that?

[zhiltsov-max]

Hi! I was able to reproduce the problem. It seems, that the SegmentationObject dir in the input archive has plain instance indices in masks. The segmentation mask 1.1 format uses PASCAL VOC masks, which are encoded differently. In this format instance masks need to use the default PASCAL VOC colormap, which can include up to 256 instances per image. You can apply this colormap manually using numpy, or you can use the following code snippet based on Datumaro (requires pip install datumaro[default]):

import datumaro as dm
import datumaro.plugins.voc_format
import datumaro.plugins.voc_format.format

inst_cm = dm.plugins.voc_format.format.VocInstColormap

def convert_mask(input_path, output_path):
    mask = dm.util.mask_tools.load_mask(input_path)[:,:,0]
    mask_in_color = dm.util.mask_tools.paint_mask(mask, inst_cm)
    dm.util.image.save_image(output_path, mask_in_color)

for path in [
    'path1.png',
    'path2.png',
    ... <your code here>
]:
    convert_mask(path, path + '-in-color')

After conversion, I was able to import your annotations without problems.

[bisnow33]

Hi thanks a lot the conversion works well ! Just a question about the result after upload polygons inside others. I can see extra labels ( for example : vegetation1 10 on right image) corresponding to background in the object image example :

Is it normal or did i miss something on the format ?

Best

[zhiltsov-max]

Hi,

Unfortunately, CVAT now has limitations on the holes in masks, it is described here: https://opencv.github.io/cvat/docs/manual/advanced/annotation-with-polygons/creating-mask/ . Basically, the imported annotations have "background" polygons in the same places as holes to represent them, and they should have bigger z-index than the underlying shapes. Probably, we need to revisit and improve this functionality now, as we have introduced (#4543) the mask support. Probably, we need to keep this issue as a feature request.

[bisnow33]

Thanks for your explanation. Not sure to understand all the problematic of the z-index but i will investigate.

[nmanovic]

@bisnow33 , we have added brush tool: https://opencv.github.io/cvat/docs/manual/advanced/annotation-with-brush-tool/