search

facebookresearch / Ego4d

Ego4d dataset repository. Download the dataset, visualize, extract features & example usage of the dataset
https://ego4d-data.org/docs/
MIT License
361 stars 49 forks source link

Missing videos when downloading EgoExo4D dataset #362

Open dhkim0225 opened 2 days ago

dhkim0225 commented 2 days ago

Hello,

I'm building the TokenBench dataset, which includes 200 videos from the EgoExo4D project.

I've been using the Ego4D CLI to download both v1 and v2 videos. However, I've encountered an issue where several videos are unavailable in my downloaded dataset.

The following videos are missing:

/mnt/cnsmldb108/user/egoexo4d/takes/cmu_bike01_3/frame_aligned_videos/aria01_214-1.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/cmu_bike01_3/frame_aligned_videos/cam01.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/cmu_bike15_1/frame_aligned_videos/aria01_214-1.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/cmu_bike15_1/frame_aligned_videos/cam01.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/cmu_soccer14_3/frame_aligned_videos/aria01_214-1.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/cmu_soccer14_3/frame_aligned_videos/cam01.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/georgiatech_covid_06_11/frame_aligned_videos/aria01_214-1.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/georgiatech_covid_06_11/frame_aligned_videos/cam01.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/iiith_cooking_100_6/frame_aligned_videos/aria01_214-1.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/iiith_cooking_100_6/frame_aligned_videos/cam01.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/indiana_music_01_5/frame_aligned_videos/aria01_214-1.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/indiana_music_01_5/frame_aligned_videos/cam01.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/nus_cpr_26_4/frame_aligned_videos/aria01_214-1.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/nus_cpr_26_4/frame_aligned_videos/cam01.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/sfu_cooking017_8/frame_aligned_videos/aria01_214-1.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/sfu_cooking017_8/frame_aligned_videos/cam01.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/sfu_cooking022_4/frame_aligned_videos/aria01_214-1.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/sfu_cooking022_4/frame_aligned_videos/cam01.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/sfu_cooking022_8/frame_aligned_videos/aria01_214-1.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/sfu_cooking022_8/frame_aligned_videos/cam01.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/unc_soccer_09-21-23_01_21/frame_aligned_videos/aria01_214-1.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/unc_soccer_09-21-23_01_21/frame_aligned_videos/cam01.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/unc_soccer_09-22-23_01_26/frame_aligned_videos/aria01_214-1.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/unc_soccer_09-22-23_01_26/frame_aligned_videos/cam01.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/unc_soccer_09-22-23_02_14/frame_aligned_videos/aria01_214-1.mp4
/mnt/cnsmldb108/user/egoexo4d/takes/unc_soccer_09-22-23_02_14/frame_aligned_videos/cam01.mp4

I've checked the Ego4D dataset viewer, and it appears that at least some of these videos (e.g., cmu_bike01_3) exist in the dataset.

Could you please provide guidance on how to download these missing videos? Are they part of a different subset, or do they require a separate download process?

Your assistance in resolving this issue would be greatly appreciated, as it's crucial for completing the TokenBench dataset.

Thank you for your time and support.

Best regards, Donghyun Kim

(panda70m) ➜  user sudo -H $(which egoexo) -o /mnt/cnsmldb108/user/egoexo4d --num_workers 20 --release v2 --parts takes

Obtaining part metadata ...
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [02:26<00:00, 146.45s/it]
Done
Checking current download status ...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50383/50383 [00:06<00:00, 7853.64it/s]
Downloaded: 100.000% = 9828.700GiB / 9828.700GiB (50383 / 50383 files) downloaded
Everything has been downloaded. Bye.
(panda70m) ➜  user sudo -H $(which egoexo) -o /mnt/cnsmldb108/user/egoexo4d --num_workers 20 --release v1 --parts takes

Obtaining part metadata ...
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:18<00:00, 18.14s/it]
Done
Determining what to download ...
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 36805/36805 [08:48<00:00, 69.63it/s]
Checking current download status ...
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 36805/36805 [00:02<00:00, 13734.95it/s]
Downloaded: 100.000% = 7969.379GiB / 7969.379GiB (36805 / 36805 files) downloaded
Everything has been downloaded. Bye.
(panda70m) ➜  user ls /mnt/cnsmldb108/user/egoexo4d/takes/cmu_bike01_3/                                                
ls: cannot access '/mnt/cnsmldb108/user/egoexo4d/takes/cmu_bike01_3/': No such file or directory
(panda70m) ➜  user ls /mnt/cnsmldb108/user/egoexo4d/takes/cmu_bike01_2 
aria01_noimagestreams.vrs  ego_preview.mp4  frame_aligned_videos  trajectory