search

MichiganCOG / A2CL-PT

Adversarial Background-Aware Loss for Weakly-supervised Temporal Activity Localization (ECCV 2020)
MIT License
46 stars 8 forks source link

Fail to reproduce the reported performance #1

Closed staceycy closed 3 years ago

staceycy commented 4 years ago

Hi,

Thank you very much for your interesting work.

I have run your code with the default parameters, but fail to get the reported results in the paper. Following are the results on validation set: 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 My results 0: 26.37 || 98.25 | 56.94, 51.97, 42.12, 33.52, 25.24, 15.52, 7.76, 3.62, 0.66 Reported results 0: 29.99 || 98.39 | 61.17, 56.08, 48.05, 38.98, 30.13, 19.15, 10.55, 4.81, 0.95

Could you please tell me whether I need to modify any other parameters in your codes?

Thank you very much.

Best, Stacey

kylemin commented 4 years ago

Hi,

Thank you for your interest. It might be due to different system specifications. Could you let me know the information of your OS, GPU, and PyTorch version? I do not think that you need to modify any other parameters than the random seed (args.seed in main.py). I recommend to test with other seed numbers and repeat the training multiple times.

Thank you.

staceycy commented 4 years ago

Hi @kylemin,

Thank you very much for your reply. After changing the Pytorch version, I can achieve similar results as yours. By the way, is it possible for you to share the feature and annotation for the ActivityNet dataset?

Thank you.

Stacey

kylemin commented 4 years ago

Hi again, I am glad that you could achieve similar results. Sure, this is the link to the ActivityNet features we used. ActivityNet-features I also uploaded the annotation.json file in the google drive. Thank you. Kyle

staceycy commented 4 years ago

Hi Kyle,

Many thanks for your prompt reply. I will try with the ActivityNet features and get back to you when the training is done.

Thank you again for your kind help.

Best, Stacey