v7labs / darwin-py

Library and commandline tool for managing datasets on darwin.v7labs.com
MIT License
115 stars 42 forks source link

[DAR-2981][External] Catch & handle blocking errors / non-blocking warnings when importing annotations to slots #895

Closed JBWilkie closed 2 months ago

JBWilkie commented 3 months ago

Problem

When importing annotations to multi-slotted or multi-channeled items, there are a few ways users can go wrong that we want to start handling:

Multi-slotted items: All annotations that are not linked to a slot are imported to the default slot, which is the 1st slot listed under slots in the List Items API response. There is no feedback that this is happening

Multi-channeled items: All annotations that are not linked to a slot are also imported to the default slot, which is the same as above. However, multi-channeled items require that all of their annotations exist in the base slot, which is not necessarily the same as the default slot. The base slot is the first slot listed under layout.slots_grid in the List Items API response. If annotations are uploaded to a non-base slot, they will not be visible in the workview

Solution

To assist users when they run into these situations, we introduce:

Warnings and errors accumulate during the import process, and are displayed afterwards. All files except those that generate at least one error are imported. A sample warning looks like this:

WARNING: Although they imported successfully, the following 2 file(s) had the following non-blocking warnings:
- File: /Users/john/Desktop/import/cars-backwards-slots.json, warnings:
  - Annotation imported to multi-slotted item /dir1/cars-backwards-slots not assigned slot. Uploading to the default slot: 1
  - Annotation imported to multi-slotted item /dir1/cars-backwards-slots not assigned slot. Uploading to the default slot: 1
- File: /Users/john/Desktop/import/cars-channels_2.json, warnings:
  - Annotation imported to multi-channeled item /dir2/cars-channels not assigned a slot. Uploading to the base slot: 3
  - Annotation imported to multi-channeled item /dir2/cars-channels not assigned a slot. Uploading to the base slot: 3
WARNING: 1 file(s) have the following blocking issues and could not be imported. Please resolve these issues and re-import them.
- File: /Users/john/Desktop/import/cars-channels.json, errors:
  - Annotation linked to slot 4 of the multi-channeled item /dir2/cars-channels. Annotations uploaded to multi-channeled items have to be uploaded to the base slot, 
which for this item is 3.
  - Annotation linked to slot 1 of the multi-channeled item /dir2/cars-channels. Annotations uploaded to multi-channeled items have to be uploaded to the base slot, 
which for this item is 3.

Finally, if users attempt to import annotations to multi-slotted or multi-channeled items with a format other than Darwin JSON 2.0, we raise a TypeError. This is because only Darwin JSON 2.0 supports multi-slotted & multi-channeled items. For example:

raise TypeError(
TypeError: You are attempting to import annotations to multi-slotted or multi-channeled items using an annotation format (coco) that doesn't support them. To import annotations to multi-slotted or multi-channeled items, please use the Darwin JSON 2.0 format: https://docs.v7labs.com/reference/darwin-json

Changelog

Introduced non-blocking warnings & blocking errors when importing annotations to multi-slotted and multi-channeled items. If errors or warnings are encountered, we provide feedback on how to resolve the errors

linear[bot] commented 3 months ago

DAR-2981 darwin-py: consider channel annotation upload behaviour

JBWilkie commented 2 months ago

Closing in favour of a new approach in this PR