search

TengdaHan / CoCLR

[NeurIPS'20] Self-supervised Co-Training for Video Representation Learning. Tengda Han, Weidi Xie, Andrew Zisserman.
Apache License 2.0
286 stars 32 forks source link

what is the video_source.json in /dataset/lmdb_dataset.py #7

Closed junmin98 closed 3 years ago

junmin98 commented 3 years ago

First of all, thank you so much for sharing source code.

With the newly updated code, I'm going to run the code for ucf101 and hbdb51. I have extracted the frame of dataset and converted it to .lmdb (I got ucf101_frame.lmdb and ucf101_frame.lmdb-order)

So I tried to evaluate with the pre-trained model you uploaded.

But, I got the error: FileNotFoundError: [Error 2] No such file or directory: 'my path/dataset/ucf101/video_source.json'

Can you tell me what 'video_source.json' is and how to get it?

Thank you!

TengdaHan commented 3 years ago

Sorry, it was my fault. I will update this part of the code later.

for UCF101, this video_source.json is a dictionary of source_name: source_id, like:

{"v_ApplyEyeMakeup_g01": 0, 
"v_ApplyEyeMakeup_g02": 1, 
"v_ApplyEyeMakeup_g03": 2,
...
"v_YoYo_g25": 2524}

The order doesn't matter, as long as you save it in json, and always use the same json file.

The purpose is to handle the "sibling videos" in UCF101 (and HMDB51) dataset, where the given 13K video clips are actually cropped from only 2.5K source videos. For InfoNCE (instance-discrimination like loss), isolating the 'sibling videos' makes more sense.

This process is not necessary, it's just for my ablation study in experiments. Also not necessary of Kinetics as each video is from a unique source.

You can create such a json file to continue. Or you can directly load the frames like https://github.com/TengdaHan/DPC/blob/master/dpc/dataset_3d.py, there is no difference.

junmin98 commented 3 years ago

thank you!