search

allenai / sherlock

Code, data, models for the Sherlock corpus
Apache License 2.0
55 stars 7 forks source link

Multiple bounding box annotation #5

Closed Yangyi-Chen closed 1 year ago

Yangyi-Chen commented 1 year ago

Hi,

Thanks for your great work. When I download the evaluation dataset, I find some instances have multiple bounding boxes annotations, for example: {"inputs": {"image": {"url": "http://s3-us-west-2.amazonaws.com/ai2-rowanz/vcr1images/movieclips_Firewalker/Tv6tdiEMGaY@38.jpg", "width": 1280, "height": 707}, "bboxes": [{"height": 137, "width": 143, "left": 751, "top": 163}, {"height": 142, "width": 101, "left": 644, "top": 111}, {"height": 157, "width": 415, "left": 67, "top": 131}], "clue": "men wearing cowboy hats", "confidence": 2.0, "obs_idx": 1}, "targets": {"inference": "the men are ranchers"}, "instance_id": "a368d19e921b4af84e9601abc999cf41", "split_idx": 19}

So how can I decide which "bboxe" the clue & inference are actually referring to? Thanks!

jmhessel commented 1 year ago

Hi @Yangyi-Chen ! Thanks for your interest in our work. As described in the paper, we allow annotators to specify multiple bounding box for each clue/inference. In this case, the clue is the cowboy hats, and I expect that the multiple bounding boxes are on several instances of cowboy hats.

Yangyi-Chen commented 1 year ago

Thanks for the clarification!