search

EGO4D / audio-visual

MIT License
59 stars 9 forks source link

Disconnected track annotation #5

Open kylemin opened 2 years ago

kylemin commented 2 years ago

Annotations for some tracks are weird. For example, track_2006 of 49c27d75-44fa-4947-899a-8d0511a10416 (clip) has only 12 frames but its frame footprint is ridiculously long as below:

1st frame: {'x': 651.16, 'y': 279.9, 'width': 55.22, 'height': 49.51, 'frame': 1268, 'video_frame': 14228, 'clip_frame': None}
...
12th (last) frame: {'x': 1662.23, 'y': 0.0, 'width': 253.24, 'height': 266.57, 'frame': 4016, 'video_frame': 16976, 'clip_frame': None}

It has 12 frames but the first clip_frame is 1268 and the 12th clip_frame is 4016. How can I use this track? Is it an annotation noise?

Interesting thing is that track_2008 of the same clip (person_id of track_2006 and track_2008 is identical) looks like as below..

1st frame: {'x': 788.25, 'y': 512.2, 'width': 66.64, 'height': 64.74, 'frame': 1433, 'video_frame': 14393, 'clip_frame': None}
...
89th (last) frame: {'x': 811.1, 'y': 5.71, 'width': 190.41, 'height': 119.96, 'frame': 4490, 'video_frame': 17450, 'clip_frame': None}

I cannot understand the two tracks. They are for same person on the same clip.

seattlesunshine commented 2 years ago

Possibly some annotation noise. I will look into this.

sanjayss34 commented 2 years ago

I'm also confused about the annotation format. Specifically, I see that two different "tracking paths" for the same "person" can actually be two different people (e.g. in frame 900 of clip 56c5af79-f9d4-478d-96ef-6d71e0bbbdfe.mp4 , track IDs track_0 and track_4) . Could you please clarify what "tracking paths" are and whether everything inside data['videos'][i]['clips'][j]['persons'][k] corresponds to a single person?

ebyrne commented 2 years ago

Apologies on the delay. We can confirm there's an issue here, it appears that two tracks have been combined. (Note the discontinuity in frame numbers halfway through the paths - as you suspect, tracks should certainly be continuous.)

We'll have an update shortly on this and we'll report back here, and will likely roll any corrections into the v1.1 update coming shortly: https://ego4d-data.org/docs/data/updates/

ebyrne commented 2 years ago

Just to follow up here, the release will be available in roughly two weeks. If you need the corrections prior to that or are interested in testing them upfront, please reach out. Otherwise please to to the forum and https://ego4d-data.org/docs/data/updates/ for the update. Appreciate the issue and your patience here!