linjieli222 / HERO

Research code for EMNLP 2020 paper "HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training"
https://arxiv.org/abs/2005.00200
MIT License
230 stars 34 forks source link

Datapath #44

Open avinashsai opened 2 years ago

avinashsai commented 2 years ago

Hi, Congrats on the amazing work.

I downloaded data using bash scripts/download_tvr.sh $PATH_TO_STORAGE (I gave my own $PATH_TO_STORAGE). How can I mention this data path in the fine-tuning script both single gpu and 8 gpus?

Thank you.

linjieli222 commented 2 years ago

Thanks for your interests in this project. Please take a look at https://github.com/linjieli222/HERO#quick-start. When launching docker container, you will need to point to $PATH_TO_STORAGE

avinashsai commented 2 years ago

Hi, Thanks for your reply. I want to run outside the docker. Can you point to the path change for it?

linjieli222 commented 2 years ago

Take look at this file https://github.com/linjieli222/HERO/blob/32c1c523c7a9f547a29f14c8e33dec24ebd14156/launch_container.sh#L23

Basically, we map (1) $PATH_TO_STORAGE/txt_db to /txt inside the docker (2) $PATH_TO_STORAGE/video_db to /video inside the docker (3) $PATH_TO_STORAGE/finetune to /storage inside the docker (4) $PATH_TO_STORAGE/pretrained to /pretrain inside the docker

You can change the finetuning command accordingly.

avinashsai commented 2 years ago

Thanks for your response. I am using single GPU instead of 8 GPUs. What are the changes to be made in the config file eg: train-tvc-8gpu.json to make sure that I am training for the same number of epochs, training samples on a single GPU?

linjieli222 commented 2 years ago

The simple answer is 8 x the default value in config.

However, it is highly possible that your single-GPU finetuning cannot match our 8-GPU finetuning result. Check this thread on why: https://github.com/linjieli222/HERO/issues/14#issuecomment-795040881