clamsproject / aapb-annotations

Repository to store manual annotation dataset developed for CLAMS-AAPB collaboration
3 stars 0 forks source link

Annotation containment convention #42

Closed keighrim closed 10 months ago

keighrim commented 10 months ago

In a recent discussion, we started to think about conventions for annotating object boundaries.

For example; here's a question regarding TimeFrame annotations. Does the annotated interval contain the target interval? Or does the target interval contain the annotated interval?

For what's been done in annotation projects from 2022, I believe (CMIIW @jarumihooi ) in all cases of TimeFrame annotations, the annotator only marked "inner" boundaries, ignoring any transitioning effects. (for instance, see this documentation for chyron annotation https://github.com/clamsproject/aapb-annotations/blob/5a19f00bce6700417665e033fdde8b920923af03/newshour-chyron/readme.md#what-to-annotate) So we can continue doing so to keep consistency. Namely, we can say, for TimeFrame "gold" data in this repository, that the annotation is narrower than the target feature.

More questions raised during the discussion, mostly for moving visual features. For example; rolling credit (characters only partially rendered when it starts and ends rolling), camera panning over objects such as "cat" (objects only partially shown in the camera frame when they come in and go out of the frame), and there's combination of both (camera panning over moving objects). A quick answer we drew was "it depends" (e.g., It depends on your target phenomenon. If the relevant target is “cat part”, then yes transition should be covered in the annotation. If the relevant target is “cat”, then no, only when "whole cat" appeared should be annotated).

Also note that, for spatial media, the convention may be the opposite. The annotation (like the bounding box) is wider/broader than the content of interest.

jarumihooi commented 10 months ago

Has this issue been resolved by the repo-level-conventions explanation?

Went further and explained that sometimes annotation is subinterval, sometimes it is not.

keighrim commented 10 months ago

I agree, fixed via e48dc4e0b806d893fd7e5f59212b0f0cfe791e42.