JonghwanMun / LGI4temporalgrounding

Repository for the CVPR-20 paper "Local-Global Video-Text Interactions for Temporal Grounding"
129 stars 17 forks source link

About I3D feature #21

Open xljh0520 opened 2 years ago

xljh0520 commented 2 years ago

Hi, thanks for releasing your code! I have a question about how you extract the feature in Charades dataset with I3D model. Could you help me with it? Thanks in advance!

JonghwanMun commented 2 years ago

I used the pre-trained model (i.e., the models/rgb_charades.pt from https://github.com/piergiaj/pytorch-i3d), so I did not fine-tune I3D model by myself.

I think you can fine-tune I3D model by following the instruction in https://github.com/piergiaj/pytorch-i3d.

xljh0520 commented 2 years ago

I used the pre-trained model (i.e., the models/rgb_charades.pt from https://github.com/piergiaj/pytorch-i3d), so I did not fine-tune I3D model by myself.

I think you can fine-tune I3D model by following the instruction in https://github.com/piergiaj/pytorch-i3d.

Thanks for your reply! I still have some questions about the detail of the feature extractor.

  1. Did you use the code from pytorch-i3d to extract the feature of Charades videos?
  2. If so, what is the hyperparameter setting, such as fps and stride? Because I tried to extract the feature in another video and I think it's best to keep the feature extractor hyperparameter the same as yours.

Thanks in advance!

JonghwanMun commented 2 years ago

I am not sure, but, I used pytorch-i3d following ExCL: Extractive Clip Localization Using Natural Language Descriptions

I think, following the paper, I would use 5 fps (or 30 fps). For stride, I think you can compute it reversely from extracted I3D features of Charades. For extraction code, I used the code from pytorch-i3d after a minor modification.

xljh0520 commented 2 years ago

I am not sure, but, I used pytorch-i3d following ExCL: Extractive Clip Localization Using Natural Language Descriptions

I think, following the paper, I would use 5 fps (or 30 fps). For stride, I think you can compute it reversely from extracted I3D features of Charades. For extraction code, I used the code from pytorch-i3d after a minor modification.

Could you share the code you modified? Thanks a lot!

JonghwanMun commented 2 years ago

Sorry that I do not have the modified code.