Open thefatbandit opened 2 years ago
Generated data-set using the yt_scraping script: Drive
400 and 600 video directories are present respectively in kinetics400 and kinetics600 directories named after the labels. Each video directory contains 2 video files of 20 seconds each.
Some of the video files failed to download during the process, after giving the following error:
[download] Downloading video 2 of 518
[youtube] P2FjnKFyDtM: Downloading webpage
[youtube] P2FjnKFyDtM: Downloading android player API JSON
[youtube] P2FjnKFyDtM: Downloading MPD manifest
[youtube] P2FjnKFyDtM: Downloading MPD manifest
[info] P2FjnKFyDtM: Downloading 1 format(s): 303+251
[download] Destination: data/kinetics600/waving_hand/Waving hand coral [P2FjnKFyDtM].webm
https://manifest.googlevideo.com/api/manifest/dash/expire/1646072703/ei/H78cYs6nAYOH_tcPu4q5oAQ/ip/35.188.0.204/id/3f61639ca1720ed3/source/youtube/requiressl/yes/playback_host/rr1---sn-qxoedn7k.googlevideo.com/mh/qj/mm/31%2C26/mn/sn-qxoedn7k%2Csn-5go7ynld/ms/au%2Conr/mv/m/mvi/1/pl/20/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/initcwndbps/912500/vprv/1/mt/1646050850/fvip/5/keepalive/yes/fexp/24001373%2C24007246/itag/0/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Chfr%2Cas%2Cvprv%2Citag/sig/AOq0QJ8wRgIhAJ-WbGDprUKswdkJLkkvaYXDbkD4w2i4SYDsH65Dy2vJAiEA4Gwe6uOaRypQKzixIR7UQcvvwa097EjkIBw9xT9SmpY%3D/lsparams/playback_host%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps/lsig/AG3C_xAwRAIgRhNLb1mCg0QbOJjC4V7rRcQ7B3o3HPB-3RfSojZ3tiACIEge7ZpK7_FJiYCenQ23Y4W6eWJD9ovJY49crCBGkR6o:2: parser error : Couldn't find end of Start Tag Segmen line 2
><SegmentURL media="sq/2/dur/5.600"/><SegmentURL media="sq/3/dur/5.600"/><Segmen
^
https://manifest.googlevideo.com/api/manifest/dash/expire/1646072703/ei/H78cYs6nAYOH_tcPu4q5oAQ/ip/35.188.0.204/id/3f61639ca1720ed3/source/youtube/requiressl/yes/playback_host/rr1---sn-qxoedn7k.googlevideo.com/mh/qj/mm/31%2C26/mn/sn-qxoedn7k%2Csn-5go7ynld/ms/au%2Conr/mv/m/mvi/1/pl/20/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/initcwndbps/912500/vprv/1/mt/1646050850/fvip/5/keepalive/yes/fexp/24001373%2C24007246/itag/0/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Chfr%2Cas%2Cvprv%2Citag/sig/AOq0QJ8wRgIhAJ-WbGDprUKswdkJLkkvaYXDbkD4w2i4SYDsH65Dy2vJAiEA4Gwe6uOaRypQKzixIR7UQcvvwa097EjkIBw9xT9SmpY%3D/lsparams/playback_host%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps/lsig/AG3C_xAwRAIgRhNLb1mCg0QbOJjC4V7rRcQ7B3o3HPB-3RfSojZ3tiACIEge7ZpK7_FJiYCenQ23Y4W6eWJD9ovJY49crCBGkR6o:2: parser error : Premature end of data in tag SegmentList line 2
><SegmentURL media="sq/2/dur/5.600"/><SegmentURL media="sq/3/dur/5.600"/><Segmen
^
https://manifest.googlevideo.com/api/manifest/dash/expire/1646072703/ei/H78cYs6nAYOH_tcPu4q5oAQ/ip/35.188.0.204/id/3f61639ca1720ed3/source/youtube/requiressl/yes/playback_host/rr1---sn-qxoedn7k.googlevideo.com/mh/qj/mm/31%2C26/mn/sn-qxoedn7k%2Csn-5go7ynld/ms/au%2Conr/mv/m/mvi/1/pl/20/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/initcwndbps/912500/vprv/1/mt/1646050850/fvip/5/keepalive/yes/fexp/24001373%2C24007246/itag/0/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Chfr%2Cas%2Cvprv%2Citag/sig/AOq0QJ8wRgIhAJ-WbGDprUKswdkJLkkvaYXDbkD4w2i4SYDsH65Dy2vJAiEA4Gwe6uOaRypQKzixIR7UQcvvwa097EjkIBw9xT9SmpY%3D/lsparams/playback_host%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps/lsig/AG3C_xAwRAIgRhNLb1mCg0QbOJjC4V7rRcQ7B3o3HPB-3RfSojZ3tiACIEge7ZpK7_FJiYCenQ23Y4W6eWJD9ovJY49crCBGkR6o:2: parser error : Premature end of data in tag Representation line 2
><SegmentURL media="sq/2/dur/5.600"/><SegmentURL media="sq/3/dur/5.600"/><Segmen
^
https://manifest.googlevideo.com/api/manifest/dash/expire/1646072703/ei/H78cYs6nAYOH_tcPu4q5oAQ/ip/35.188.0.204/id/3f61639ca1720ed3/source/youtube/requiressl/yes/playback_host/rr1---sn-qxoedn7k.googlevideo.com/mh/qj/mm/31%2C26/mn/sn-qxoedn7k%2Csn-5go7ynld/ms/au%2Conr/mv/m/mvi/1/pl/20/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/initcwndbps/912500/vprv/1/mt/1646050850/fvip/5/keepalive/yes/fexp/24001373%2C24007246/itag/0/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Chfr%2Cas%2Cvprv%2Citag/sig/AOq0QJ8wRgIhAJ-WbGDprUKswdkJLkkvaYXDbkD4w2i4SYDsH65Dy2vJAiEA4Gwe6uOaRypQKzixIR7UQcvvwa097EjkIBw9xT9SmpY%3D/lsparams/playback_host%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps/lsig/AG3C_xAwRAIgRhNLb1mCg0QbOJjC4V7rRcQ7B3o3HPB-3RfSojZ3tiACIEge7ZpK7_FJiYCenQ23Y4W6eWJD9ovJY49crCBGkR6o:2: parser error : Premature end of data in tag AdaptationSet line 2
><SegmentURL media="sq/2/dur/5.600"/><SegmentURL media="sq/3/dur/5.600"/><Segmen
^
https://manifest.googlevideo.com/api/manifest/dash/expire/1646072703/ei/H78cYs6nAYOH_tcPu4q5oAQ/ip/35.188.0.204/id/3f61639ca1720ed3/source/youtube/requiressl/yes/playback_host/rr1---sn-qxoedn7k.googlevideo.com/mh/qj/mm/31%2C26/mn/sn-qxoedn7k%2Csn-5go7ynld/ms/au%2Conr/mv/m/mvi/1/pl/20/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/initcwndbps/912500/vprv/1/mt/1646050850/fvip/5/keepalive/yes/fexp/24001373%2C24007246/itag/0/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Chfr%2Cas%2Cvprv%2Citag/sig/AOq0QJ8wRgIhAJ-WbGDprUKswdkJLkkvaYXDbkD4w2i4SYDsH65Dy2vJAiEA4Gwe6uOaRypQKzixIR7UQcvvwa097EjkIBw9xT9SmpY%3D/lsparams/playback_host%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps/lsig/AG3C_xAwRAIgRhNLb1mCg0QbOJjC4V7rRcQ7B3o3HPB-3RfSojZ3tiACIEge7ZpK7_FJiYCenQ23Y4W6eWJD9ovJY49crCBGkR6o:2: parser error : Premature end of data in tag Period line 2
><SegmentURL media="sq/2/dur/5.600"/><SegmentURL media="sq/3/dur/5.600"/><Segmen
^
https://manifest.googlevideo.com/api/manifest/dash/expire/1646072703/ei/H78cYs6nAYOH_tcPu4q5oAQ/ip/35.188.0.204/id/3f61639ca1720ed3/source/youtube/requiressl/yes/playback_host/rr1---sn-qxoedn7k.googlevideo.com/mh/qj/mm/31%2C26/mn/sn-qxoedn7k%2Csn-5go7ynld/ms/au%2Conr/mv/m/mvi/1/pl/20/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/initcwndbps/912500/vprv/1/mt/1646050850/fvip/5/keepalive/yes/fexp/24001373%2C24007246/itag/0/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Chfr%2Cas%2Cvprv%2Citag/sig/AOq0QJ8wRgIhAJ-WbGDprUKswdkJLkkvaYXDbkD4w2i4SYDsH65Dy2vJAiEA4Gwe6uOaRypQKzixIR7UQcvvwa097EjkIBw9xT9SmpY%3D/lsparams/playback_host%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps/lsig/AG3C_xAwRAIgRhNLb1mCg0QbOJjC4V7rRcQ7B3o3HPB-3RfSojZ3tiACIEge7ZpK7_FJiYCenQ23Y4W6eWJD9ovJY49crCBGkR6o:2: parser error : Premature end of data in tag MPD line 2
><SegmentURL media="sq/2/dur/5.600"/><SegmentURL media="sq/3/dur/5.600"/><Segmen
^
[dash @ 0x561353f0e000] Unable to parse 'https://manifest.googlevideo.com/api/manifest/dash/expire/1646072703/ei/H78cYs6nAYOH_tcPu4q5oAQ/ip/35.188.0.204/id/3f61639ca1720ed3/source/youtube/requiressl/yes/playback_host/rr1---sn-qxoedn7k.googlevideo.com/mh/qj/mm/31%2C26/mn/sn-qxoedn7k%2Csn-5go7ynld/ms/au%2Conr/mv/m/mvi/1/pl/20/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/initcwndbps/912500/vprv/1/mt/1646050850/fvip/5/keepalive/yes/fexp/24001373%2C24007246/itag/0/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Chfr%2Cas%2Cvprv%2Citag/sig/AOq0QJ8wRgIhAJ-WbGDprUKswdkJLkkvaYXDbkD4w2i4SYDsH65Dy2vJAiEA4Gwe6uOaRypQKzixIR7UQcvvwa097EjkIBw9xT9SmpY%3D/lsparams/playback_host%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps/lsig/AG3C_xAwRAIgRhNLb1mCg0QbOJjC4V7rRcQ7B3o3HPB-3RfSojZ3tiACIEge7ZpK7_FJiYCenQ23Y4W6eWJD9ovJY49crCBGkR6o' - missing root node
https://manifest.googlevideo.com/api/manifest/dash/expire/1646072703/ei/H78cYs6nAYOH_tcPu4q5oAQ/ip/35.188.0.204/id/3f61639ca1720ed3/source/youtube/requiressl/yes/playback_host/rr1---sn-qxoedn7k.googlevideo.com/mh/qj/mm/31%2C26/mn/sn-qxoedn7k%2Csn-5go7ynld/ms/au%2Conr/mv/m/mvi/1/pl/20/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/initcwndbps/912500/vprv/1/mt/1646050850/fvip/5/keepalive/yes/fexp/24001373%2C24007246/itag/0/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Chfr%2Cas%2Cvprv%2Citag/sig/AOq0QJ8wRgIhAJ-WbGDprUKswdkJLkkvaYXDbkD4w2i4SYDsH65Dy2vJAiEA4Gwe6uOaRypQKzixIR7UQcvvwa097EjkIBw9xT9SmpY%3D/lsparams/playback_host%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps/lsig/AG3C_xAwRAIgRhNLb1mCg0QbOJjC4V7rRcQ7B3o3HPB-3RfSojZ3tiACIEge7ZpK7_FJiYCenQ23Y4W6eWJD9ovJY49crCBGkR6o: Invalid data found when processing input
ERROR: ffmpeg exited with code 1
Therefore very few of the label directories are incomplete.
Mradul