Closed jasonppy closed 1 year ago
We did not use any keywords to construct the dataset. We filtered the original datasets by using a similarity threshold between the reference transcripts and the image caption as mentioned in the paper. We downloaded each of the youtube link using the youtube-dl toolkit which has the segmented timestamps, we encourage you to read the paper for further details, which is mentioned in the dataset retrieval section.
Thanks for your reply!
Regarding the keywords for constructing the dataset. In your paper, you describe the dataset collection as
The datasets were obtained from mainly two sources, the how2 dataset and the youtube videos. The how2 dataset consists of 300h of videos with annotated transcripts in English which resulted in 220,000 samples. The youtube videos were collected using the Youtube-DL toolkit, where each of these videos had annotated transcripts and audio in English. The youtube videos accounted for 2.5 million samples of data with annotated transcripts
The youtube video part (although How2 is also from youtube, I pointed out here just to avoid confusion to other readers) is first selected via some method right? You can't just collect all the videos on the web. In your previous commit, you documented on searching youtube videos with keywords https://github.com/VanyaBK/visual_ASR_EC/tree/5a86c37e6cac0339dbfc9a2a073c06c654dc416b/youtube-downloader#scripts, which is why I inferred that you used this approach for constructing the dataset.
Thats's right, the keywords were used only to obtain the links, which is listed in the youtube_links file now. The youtube_links file do not contain the how2 data, this was obtained by contacting the author of the how2 paper.
How do you obtain the keywords?
The keywords used are the categories of the how2100M dataset
got it, thanks for your patience!
Could you also release the list of search keywords that you used for constructing the dataset?
Also, I'd appreciate it if you could release the final dataset (e.g. in the form of vid and segment timestamps) that you used for the error correction experiments
Thanks