voxel51 / fiftyone

Refine high-quality datasets and visual AI models
https://fiftyone.ai
Apache License 2.0
8.89k stars 563 forks source link

Skip samples for which metadata cannot be computed when annotating with CVAT #1624

Open ehofesmann opened 2 years ago

ehofesmann commented 2 years ago

The upload to CVAT fails if some samples cannot have metadata computed for them. It would be best if these samples could be gracefully skipped so that the samples with metadata are still able to be uploaded:

Example Error:

Uploading samples to CVAT...
Computing image metadata...
 100% |█████████████████████████████████████████████████████████████████| 93/93 [93.8ms elapsed, 0s remaining, 991.0 samples/s] 
Failed to populate metadata on 93 samples. Use `dataset.exists("metadata", False)` to retrieve them
Uncaught exception
Traceback (most recent call last):
  File "fiftyone_download_and_upload_to_CVAT.py", line 10, in <module>
    view.annotate(anno_key, label_field="detections",label_type="detections",classes=["Bicycle","Person"])
  File "/home/shao/.local/lib/python3.8/site-packages/fiftyone/core/collections.py", line 6272, in annotate
    return foua.annotate(
  File "/home/shao/.local/lib/python3.8/site-packages/fiftyone/utils/annotations.py", line 212, in annotate
    results = anno_backend.upload_annotations(
  File "/home/shao/.local/lib/python3.8/site-packages/fiftyone/utils/cvat.py", line 3155, in upload_annotations
    results = api.upload_samples(samples, self)
  File "/home/shao/.local/lib/python3.8/site-packages/fiftyone/utils/cvat.py", line 4067, in upload_samples
    self._update_shapes_tags_tracks(
  File "/home/shao/.local/lib/python3.8/site-packages/fiftyone/utils/cvat.py", line 4873, in _update_shapes_tags_tracks
    _id_map, anno_shapes = self._create_shapes_tags_tracks(
  File "/home/shao/.local/lib/python3.8/site-packages/fiftyone/utils/cvat.py", line 5376, in _create_shapes_tags_tracks
    frame_size = (metadata.width, metadata.height)
AttributeError: 'NoneType' object has no attribute 'width'
brimoor commented 2 years ago

Alternative viewpoint: there's already an informative warning message being printed, so rely on the user to decide whether it is permissible to re-run on a view with bad samples omitted, or whether for their use case they must resolve metadata errors before re-running to upload every sample

ehofesmann commented 2 years ago

Alternative viewpoint: there's already an informative warning message being printed, so rely on the user to decide whether it is permissible to re-run on a view with bad samples omitted, or whether for their use case they must resolve metadata errors before re-running to upload every sample

I agree that this error is fairly self-explanatory. How about a general skip_failures option, it can default to False.

For reference, there can also be issues when uploading to CVAT that do not raise errors, so I am already working on a way to track which samples failed to upload due to CVAT issues. A skip_failures option would just add samples that failed to upload from the FiftyOne side to the list.