Fail to open load video clips in Kinetics

[HaihangWu]

Hi Author: When I train the model with Kinetics 400, it reports "failed to load video idx ..." and there are quite lots of such error. I can see there are these videos in my dataset but they are damaged. I download videos from https://github.com/cvdfoundation/kinetics-dataset. Did you met this before? I worry that this will hurt the performance and can not get the same performance as you. Followings are some of them:

Failed to load video idx 34319 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/vJ67OZSD_ac.mp4; trial 6 Failed to load video idx 157531 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/wKsuyr6Xk30.mp4; trial 0 Failed to load video idx 157531 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/wKsuyr6Xk30.mp4; trial 1 Failed to load video idx 157531 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/wKsuyr6Xk30.mp4; trial 2 Failed to load video idx 157531 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/wKsuyr6Xk30.mp4; trial 3 Failed to load video idx 157531 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/wKsuyr6Xk30.mp4; trial 4 Failed to load video idx 157531 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/wKsuyr6Xk30.mp4; trial 5 Failed to load video idx 157531 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/wKsuyr6Xk30.mp4; trial 6 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x557958bf0540] moov atom not found [11:32:47] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ytrQs4tSR8c.mp4, Invalid data found when processing input Failed to load video idx 144178 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ytrQs4tSR8c.mp4; trial 0 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x557958bf0540] moov atom not found [11:32:47] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ytrQs4tSR8c.mp4, Invalid data found when processing input Failed to load video idx 144178 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ytrQs4tSR8c.mp4; trial 1 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x557958bd3f40] moov atom not found [11:32:47] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ytrQs4tSR8c.mp4, Invalid data found when processing input Failed to load video idx 144178 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ytrQs4tSR8c.mp4; trial 2 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x557958c3d7c0] moov atom not found [11:32:47] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ytrQs4tSR8c.mp4, Invalid data found when processing input Failed to load video idx 144178 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ytrQs4tSR8c.mp4; trial 3 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x557958c3d7c0] moov atom not found [11:32:47] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ytrQs4tSR8c.mp4, Invalid data found when processing input Failed to load video idx 144178 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ytrQs4tSR8c.mp4; trial 4 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x557958c3d7c0] moov atom not found [11:32:47] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ytrQs4tSR8c.mp4, Invalid data found when processing input Failed to load video idx 144178 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ytrQs4tSR8c.mp4; trial 5 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x557958c3d7c0] moov atom not found [11:32:47] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ytrQs4tSR8c.mp4, Invalid data found when processing input Failed to load video idx 144178 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ytrQs4tSR8c.mp4; trial 6 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x5633e4132dc0] moov atom not found [11:32:54] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/lPEz_8qnuiY.mp4, Invalid data found when processing input Failed to load video idx 71965 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/lPEz_8qnuiY.mp4; trial 0 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x5633e4132dc0] moov atom not found [11:32:54] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/lPEz_8qnuiY.mp4, Invalid data found when processing input Failed to load video idx 71965 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/lPEz_8qnuiY.mp4; trial 1 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x5633e4132dc0] moov atom not found [11:32:54] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/lPEz_8qnuiY.mp4, Invalid data found when processing input Failed to load video idx 71965 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/lPEz_8qnuiY.mp4; trial 2 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x5633e4068480] moov atom not found [11:32:54] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/lPEz_8qnuiY.mp4, Invalid data found when processing input Failed to load video idx 71965 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/lPEz_8qnuiY.mp4; trial 3 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x5633e3fa1a80] moov atom not found [11:32:54] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/lPEz_8qnuiY.mp4, Invalid data found when processing input Failed to load video idx 71965 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/lPEz_8qnuiY.mp4; trial 4 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x5633e40fd0c0] moov atom not found [11:32:54] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/lPEz_8qnuiY.mp4, Invalid data found when processing input Failed to load video idx 71965 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/lPEz_8qnuiY.mp4; trial 5 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x5633e3fa1a80] moov atom not found [11:32:54] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/lPEz_8qnuiY.mp4, Invalid data found when processing input Failed to load video idx 71965 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/lPEz_8qnuiY.mp4; trial 6 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x561c9f6d8bc0] moov atom not found [11:33:03] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/0NdUx-KPitU.mp4, Invalid data found when processing input Failed to load video idx 118477 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/0NdUx-KPitU.mp4; trial 0 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x561c9f6d8bc0] moov atom not found [11:33:03] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/0NdUx-KPitU.mp4, Invalid data found when processing input Failed to load video idx 118477 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/0NdUx-KPitU.mp4; trial 1 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x561c9f6d8bc0] moov atom not found [11:33:03] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/0NdUx-KPitU.mp4, Invalid data found when processing input Failed to load video idx 118477 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/0NdUx-KPitU.mp4; trial 2 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x561c9f6d8bc0] moov atom not found [11:33:03] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/0NdUx-KPitU.mp4, Invalid data found when processing input Failed to load video idx 118477 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/0NdUx-KPitU.mp4; trial 3 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x561c9f6d8bc0] moov atom not found [11:33:03] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/0NdUx-KPitU.mp4, Invalid data found when processing input Failed to load video idx 118477 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/0NdUx-KPitU.mp4; trial 4 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x561c9f6f51c0] moov atom not found [11:33:03] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/0NdUx-KPitU.mp4, Invalid data found when processing input Failed to load video idx 118477 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/0NdUx-KPitU.mp4; trial 5 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x561c9f6f51c0] moov atom not found [11:33:03] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/0NdUx-KPitU.mp4, Invalid data found when processing input Failed to load video idx 118477 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/0NdUx-KPitU.mp4; trial 6 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55974087a180] moov atom not found [11:33:10] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ipc-xJ815eo.mp4, Invalid data found when processing input Failed to load video idx 207691 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ipc-xJ815eo.mp4; trial 0 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55974087a180] moov atom not found [11:33:10] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ipc-xJ815eo.mp4, Invalid data found when processing input Failed to load video idx 207691 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ipc-xJ815eo.mp4; trial 1 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55974087a180] moov atom not found [11:33:10] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ipc-xJ815eo.mp4, Invalid data found when processing input Failed to load video idx 207691 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ipc-xJ815eo.mp4; trial 2 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55974087a180] moov atom not found [11:33:10] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ipc-xJ815eo.mp4, Invalid data found when processing input Failed to load video idx 207691 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ipc-xJ815eo.mp4; trial 3 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55974087a180] moov atom not found [11:33:10] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ipc-xJ815eo.mp4, Invalid data found when processing input Failed to load video idx 207691 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ipc-xJ815eo.mp4; trial 4 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55974087a180] moov atom not found [11:33:10] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ipc-xJ815eo.mp4, Invalid data found when processing input Failed to load video idx 207691 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ipc-xJ815eo.mp4; trial 5 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55974087a180] moov atom not found [11:33:10] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ipc-xJ815eo.mp4, Invalid data found when processing input Failed to load video idx 207691 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/ipc-xJ815eo.mp4; trial 6 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x5647f2c997c0] moov atom not found [11:33:14] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/Haqw7b9U-L4.mp4, Invalid data found when processing input Failed to load video idx 125115 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/Haqw7b9U-L4.mp4; trial 0 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x5647f2db7ac0] moov atom not found [11:33:14] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/Haqw7b9U-L4.mp4, Invalid data found when processing input Failed to load video idx 125115 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/Haqw7b9U-L4.mp4; trial 1 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x5647f2c997c0] moov atom not found [11:33:14] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/Haqw7b9U-L4.mp4, Invalid data found when processing input Failed to load video idx 125115 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/Haqw7b9U-L4.mp4; trial 2 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x5647f2db7ac0] moov atom not found [11:33:14] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/Haqw7b9U-L4.mp4, Invalid data found when processing input Failed to load video idx 125115 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/Haqw7b9U-L4.mp4; trial 3 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x5647f2c997c0] moov atom not found [11:33:14] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/Haqw7b9U-L4.mp4, Invalid data found when processing input Failed to load video idx 125115 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/Haqw7b9U-L4.mp4; trial 4 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x5647f0b0ebc0] moov atom not found [11:33:14] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/Haqw7b9U-L4.mp4, Invalid data found when processing input Failed to load video idx 125115 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/Haqw7b9U-L4.mp4; trial 5 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x5647f2e48b00] moov atom not found [11:33:14] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/Haqw7b9U-L4.mp4, Invalid data found when processing input Failed to load video idx 125115 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/Haqw7b9U-L4.mp4; trial 6 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55a312741300] moov atom not found [11:33:15] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/COhmwEjKejw.mp4, Invalid data found when processing input Failed to load video idx 69917 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/COhmwEjKejw.mp4; trial 0 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55a312741300] moov atom not found [11:33:15] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/COhmwEjKejw.mp4, Invalid data found when processing input Failed to load video idx 69917 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/COhmwEjKejw.mp4; trial 1 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55a3126ec100] moov atom not found [11:33:15] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/COhmwEjKejw.mp4, Invalid data found when processing input Failed to load video idx 69917 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/COhmwEjKejw.mp4; trial 2 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55a312860880] moov atom not found [11:33:15] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/COhmwEjKejw.mp4, Invalid data found when processing input Failed to load video idx 69917 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/COhmwEjKejw.mp4; trial 3 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55a310530d00] moov atom not found [11:33:15] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/COhmwEjKejw.mp4, Invalid data found when processing input Failed to load video idx 69917 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/COhmwEjKejw.mp4; trial 4 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55a310530d00] moov atom not found [11:33:15] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/COhmwEjKejw.mp4, Invalid data found when processing input Failed to load video idx 69917 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/COhmwEjKejw.mp4; trial 5 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55a310530d00] moov atom not found [11:33:15] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/COhmwEjKejw.mp4, Invalid data found when processing input Failed to load video idx 69917 from /data/gpfs/projects/punim0512/data/kinetics-dataset/k400/train/COhmwEjKejw.mp4; trial 6

[Andy1621]

Hi! I have uploaded one copy of my Kineti400 to BaiduYun. Link: https://pan.baidu.com/s/1sOTjfryT1j6BmA9MOqbl9w Password: 6jb5

[HaihangWu]

Thanks for the link. I have downloaded the dataset from this link but I fail to decompress these files using the instructions ( (1) cat xa* >> train_k400.zip (2) unzip train_k400.zip). Instruction 1 is successful but for instruction 2, it reports "Archive: train_k400.zip End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive."

I then use the command zip -FF train_k400.zip --out Pictures_single.zip to fix the problem of train_k400.zip. But it fails and report out of memory. Could help check these? Thanks a lot

[Andy1621]

Can you try to use tar zxvf train_k400.zip? Since this dataset has been uploaded for a long time, I'm not sure whether they are a .zip file or a .tgz file.