search

facebookresearch / AVT

Code release for ICCV 2021 paper "Anticipative Video Transformer"
Apache License 2.0
152 stars 28 forks source link

cant create video dataset #30

Closed sanketsans closed 2 years ago

sanketsans commented 2 years ago

Hi, I am trying to run the code for ek100 dataset. But i notice that it is not able to compute any video clips for dataset creation on as here https://github.com/facebookresearch/AVT/blob/2d6781d5315a4c53bd059b1cd11ee46bd4427648/datasets/data.py#L41

The _dataset variable is EpicKitchen class object since the _target variable defined here is of the same type. https://github.com/facebookresearch/AVT/blob/2d6781d5315a4c53bd059b1cd11ee46bd4427648/conf/dataset/epic_kitchens100/anticipation_train.yaml#L3

Therefore it is not able to execute this line. https://github.com/facebookresearch/AVT/blob/2d6781d5315a4c53bd059b1cd11ee46bd4427648/datasets/data.py#L46 and it execute the except line after this.

Can you help me in this regard. Maybe I am doing something wrong ! Thanks :)

rohitgirdhar commented 2 years ago

Can you share what error are you getting and which config are you running?

sanketsans commented 2 years ago

I am using 01_ek100_avt.txt (https://github.com/facebookresearch/AVT/blob/main/expts/01_ek100_avt.txt) as the config. In the logs, when I check during the dataset creation ; it states dataset created with ~68k elts but no video clips found.

I have my vid directory same as yours and I changed the root dir in anticipation_train.yaml.

Besides, one more thing; during init of the model from the checkpoint as mentioned in the config file. It says https://github.com/facebookresearch/AVT/blob/2d6781d5315a4c53bd059b1cd11ee46bd4427648/func/train.py#L496

though the missing keys are {} (none I guess), but there are two unexpected_keys. Is there something wrong in here ? or is the model loaded correctly here anyway ?

Anirudh257 commented 2 years ago

@sanketsans, I also get the same error but the final results are fine.

sanketsans commented 2 years ago

@Anirudh257 Thanks for feedback. So you mean the model is loading perfectly fine even with the errors ? Also, do you also get the CUDA out of memory error ? I am trying to run this with two GPU (16 & 8GB) using -l option. Even then it is failing. Not sure, what is going wrong

Anirudh257 commented 2 years ago

Yes, I used to get it on 2 GPUs. So, I am running with more GPUs and distributing it across the nodes. You can try lowering the batch-size but the best way is to use more GPUs/more RAM.