search

26hzhang / VSLNet

Span-based Localizing Network for Natural Language Video Localization (ACL 2020)
MIT License
100 stars 17 forks source link

About the shared video features #10

Closed Raymond-sci closed 2 years ago

Raymond-sci commented 2 years ago

Hi there - many thanks for sharing the resources, it helps a lot. I noticed that there are two types of features you shared for each dataset ("new" vs "org") but I hardly find any explanation on that through your repo. Can you please give some hints on this?

26hzhang commented 2 years ago

For Charades-STA, the "new" means the I3D features fine-tuned on Charades dataset, while "org" is I3D features without fine-tuned. For ActivityNet Captions dataset, the "new" means the I3D features, while "org" is the C3D features with PCA dim reduction officially provided by ActivityNet (http://activity-net.org/challenges/2016/download.html).

Hope it helps you.

Raymond-sci commented 2 years ago

Yes, it is clear now, thanks.

Besides, in this case, I should be able to reproduce the results in the lower part of table 2 by the "new" features, shouldn't I? When I used the "new" features (fine-tuned on Charades) as the inputs to the provided pytorch implementation of VSLNet, the results on Charades (mIoU/IoU@0.3/IoU@0.5/IoU@0.7=47.33/67.26/50.46/31.53) were inferior to the ones reported (50.02/70.46/54.19/35.22). Given the gaps are nonnegligible (~4%), I am wondering if I missed any things? Or says, it is just because of the 'unknown reasons' you mentioned in README that makes the pytorch implementation worse than the TF counterpart?

26hzhang commented 2 years ago

@Raymond-sci For the PyTorch, I also feel quite puzzle about this issue, since I am also a fresh user of PyTorch. Besides, I ran the PyTorch codes on Charades-STA too, it is better than your results. Emmmm, I will try to check if there is something wrong in my codes when I am free. Sorry for the inconvenience. If you want to reproduce the results, I suggest to run the tensorflow codes first.

sLdZ0306 commented 2 years ago

Hi~ Thanks for your sharing! Could you please introduce the difference between the "org/new" for TACoS dataset?

aries-young commented 1 year ago

Hi~ Thanks for your sharing! Could you please introduce the difference between the "org/new" for TACoS dataset?

Hello, have you figure out the difference between the "org/new" for TACoS dataset?