COCO export not working for polygons with 2 points only

[stanbiryukov]

I'm unable to export annotations to COCO format on the latest master branch. Might be a pycocotools versioning issue.

Could not dump annotations for the task 4
Error: Request failed with status code 500. "cvat.apps.engine.utils.InterpreterError: Exception at line 307: input type is not supported.\n".

And here's the relevant error in docker:

cvat          | 2019-12-13 16:00:22,023 DEBG 'rqworker_default_0' stderr output:
cvat          | 16:00:22 cvat.apps.engine.utils.InterpreterError: Exception at line 307: input type is not supported.
cvat          | Traceback (most recent call last):
cvat          |   File "/home/django/cvat/apps/engine/utils.py", line 45, in execute_python_code
cvat          |     exec(source_code, global_vars, local_vars)
cvat          |   File "<string>", line 1, in <module>
cvat          |   File "<string>", line 319, in dump
cvat          |   File "<string>", line 250, in insert_annotation_data
cvat          |   File "<string>", line 141, in polygon_area_and_bbox
cvat          |   File "pycocotools/_mask.pyx", line 307, in pycocotools._mask.frPyObjects
cvat          | Exception: input type is not supported.

[nmanovic]

@stanbiryukov , Can you reproduce the problem on develop branch as well?

[stanbiryukov]

My mistake - just double checked and the above error was on the develop branch.

Master is not letting me login

Could not check authorization on the server
Error: Network Error.

[nmanovic]

@stanbiryukov , master should work fine. But let's concentrate on the issue. Can it be reproduced on a task without annotations? Is it a task with video or images?

[stanbiryukov]

Restarted docker, switched and logged in to master and getting the same error.
I'm working on an image task and confirmed that export works fine without annotations.
The issue might be a bit more complicated since I'm working with an originally uploaded COCO dataset. If I restart and create a new annotation project I'm able to export to COCO successfully.

[stanbiryukov]

Note that CVAT xml exports work just fine so this might be a pycoco related issue and not cvat.

[nmanovic]

@stanbiryukov , if you can give us some steps to reproduce the problem it will be easy for us to investigate. In any case CVAT should handle the problem gratefully.

[nmanovic]

A possible way to export the task as datumaro project and put it here. It will not have any images. Only annotations in an internal format. Probably it will give us some ideas how to reproduce.

[stanbiryukov]

Thanks - here's the datumaro export.

task_imprints-2019_12_13_16_35_05-datumaro_project.zip

[zhiltsov-max]

As far as I understand, initially you used export annotations button? Could you try using export a dataset?

[stanbiryukov]

Exporting as MS COCO dataset does not work either - same type error:

cvat          | Exception: input type is not supported.
cvat          | Traceback (most recent call last):
cvat          |   File "/usr/local/lib/python3.5/dist-packages/rq/worker.py", line 812, in perform_job
cvat          |     rv = job.perform()
cvat          |   File "/usr/local/lib/python3.5/dist-packages/rq/job.py", line 588, in perform
cvat          |     self._result = self._execute()
cvat          |   File "/usr/local/lib/python3.5/dist-packages/rq/job.py", line 594, in _execute
cvat          |     return self.func(*self.args, **self.kwargs)
cvat          |   File "/home/django/cvat/apps/dataset_manager/task.py", line 331, in export_project
cvat          |     server_url=server_url)
cvat          |   File "/home/django/cvat/apps/dataset_manager/task.py", line 221, in export
cvat          |     save_dir=save_dir, save_images=save_images)
cvat          |   File "/home/django/datumaro/datumaro/components/project.py", line 521, in export
cvat          |     converter(dataset, save_dir)
cvat          |   File "/home/django/datumaro/datumaro/components/converters/ms_coco.py", line 422, in __call__
cvat          |     converter.convert()
cvat          |   File "/home/django/datumaro/datumaro/components/converters/ms_coco.py", line 406, in convert
cvat          |     task_conv.save_annotations(item)
cvat          |   File "/home/django/datumaro/datumaro/components/converters/ms_coco.py", line 190, in save_annotations
cvat          |     rles = mask_utils.frPyObjects(segmentation, h, w)
cvat          |   File "pycocotools/_mask.pyx", line 307, in pycocotools._mask.frPyObjects
cvat          | Exception: input type is not supported.

[zhiltsov-max]

As far as I can see, the issue is here, the code fails because of there is a polygon with only 2 points, and it comes first in the array. Interesting.

[stanbiryukov]

Perhaps the best approach is to add a warning and skip those types of segments upon export or original import? I do see the 2 point polygon in the CVAT xml file now. The CVAT api correctly doesn't allow for creation of polygons with less than 3 points.

[stanbiryukov]

Looks like with the new commit and this use case, there is now an error Unable to import annotation #1113: a polygon has too few points (2) @zhiltsov-max @nmanovic

Is it possible to skip over the problematic polygons instead of breaking the entire import? In this case I have thousands of annotations that can no longer be imported because of a few polygon errors.

[nmanovic]

@stanbiryukov , let us understand better the problem. How did you get annotations with only 2 points for a polygon?

[stanbiryukov]

@nmanovic I originally used a different annotation tool with this project: https://github.com/jsbroks/coco-annotator

I suppose I can try and manually edit the JSON file, however, from a user perspective, it seems most intuitive that the import raises a warning and skips problematic polygons instead of breaking the import.

[nmanovic]

@zhiltsov-max , what do you think? What is the best way to go here from your perspective?

My thoughts: I think datumaro can have special options (e.g. --fix or --ignore-problems or --skip-issues) to import such bad annotations and export them into a valid COCO file again. I don't think that in CVAT we should ignore errors but for the case we can give a tool (datum.py) which will "filter" bad cases for the user and produces a valid COCO file which can be uploaded.

[zhiltsov-max]

The idea to provide means in Datumaro to deal with the issue sounds well for me. What could be the good strategy of handling this, just ignoring the wrong annotations?

[nmanovic]

@zhiltsov-max ,

What could be the good strategy of handling this, just ignoring the wrong annotations?

Let's ignore and print a warning on the console.

[stanbiryukov]

Hi there - any update on the raised warning strategy for converting annotations?

[zhiltsov-max]

It will be like stated above - in CVAT will be an error, in Datumaro will be an option to ignore corrupted annotations.

[zhiltsov-max]

Error display during import and export operations will be tracked in #1529

[azhavoro]

@zhiltsov-max Has this issue been resolved?

[zhiltsov-max]

Wrong annotations are not exported or imported, but there is no option to ignore them, or to print a list of errors.

[nmanovic]

@zhiltsov-max , it looks like it is something on what you are working now.