"Timed Instruction" in RxR Dataset not consistent?

google-research-datasets / RxR

Room-across-Room (RxR) is a large-scale, multilingual dataset for Vision-and-Language Navigation (VLN) in Matterport3D environments. It contains 126k navigation instructions in English, Hindi and Telugu, and 126k navigation following demonstrations. Both annotation types include dense spatiotemporal alignments between the text and the visual perceptions of the annotators

Creative Commons Attribution 4.0 International

113 stars 12 forks source link

"Timed Instruction" in RxR Dataset not consistent? #2

Closed siddk closed 3 years ago

siddk commented 3 years ago

Hi!

I've been trying to visualize the dataset via the code in visualizations/first_person.html but have run into a problem that I thought I should flag. It seems like in the original dataset (at least in rxr_train) some guide examples are missing the "start_time" and "end_time" fields in the "timed_instruction" list.

For example, here are some broken instruction_ids (from rxr_train) that I found in just trying out the existing script:

peteanderson80 commented 3 years ago

Thanks, there are a small number of words that don't have timing info. In that case it can be approximated from the neighbors. I've made a note of this in the readme.