search

ptirupat / MLAD

Implementation of paper "Modeling Multi-Label Action Dependencies for Temporal Action Localization"
49 stars 6 forks source link

get a lower fmap #6

Open mangoooooooo opened 2 years ago

mangoooooooo commented 2 years ago

hi, thanks for sharing your code. I validate the model you provided on multithumos and get 'mAP: 0.492592', it is lower than your paper. And I tried to train with your script, got mAP: 0.425716. Do you have any idea about it? Do I need to modify the script or code? Hoping for your reply.

sparkstj commented 2 years ago

Same, and for charades, I modify the num_clips in script from 128 to 64, validate the model with provided checkpoint and i3d two-stream features extracted by this repo, and got mAP 18.12%.

Any advice to improve the mAP?

Wyz2927 commented 2 years ago

hi, thanks for sharing your code. I validate the model you provided on multithumos and get 'mAP: 0.492592', it is lower than your paper. And I tried to train with your script, got mAP: 0.425716. Do you have any idea about it? Do I need to modify the script or code? Hoping for your reply.

I also get lower fmap. When I train with the given script, I got mAP:0.435. Can you reproduce this paper's results now?

dairui01 commented 2 years ago

I am also not able to reproduce the performance... with the provided feature and model on MultiTHUMOS

ptirupat commented 2 years ago

@mangoooooooo and @Wyz2927 Can you share the training log? Also, what is the PyTorch version you are using?

dairui01 commented 2 years ago

Here is my full training log on MultiTHUMOS [link] With the same parameters provided in README (Combined RGB+FLOW).

Here is the log at epoch 2490: Validation Epoch: 2490, Average Precision: [0.14744237 0.77798019 0.32363736 0.69155635 0.86624762 0.57490634 0.30397834 0.87930204 0.53156238 0.53421548 0.8728032 0.7256438 0.6828776 0.88034331 0.89258777 0.72323648 0.5267213 0.52542956 0.61803609 0.39572705 0.58717902 0.19906541 0.11727117 0.44926292 0.68296983 0.74561331 0.46085191 0.42119999 0.06038184 0.00384729 0.00435112 0.00761441 0.30809785 0.02378978 0.10185604 0.18713782 0.14313186 0.03572717 0.05492678 0.68844336 0.08151524 0.20442906 0.49038632 0.58514008 0.38297765 0.10197023 0.16036817 0.28013663 0.69343029 0.27335341 0.00124359 0.49961756 0.00127861 0.54743744 0.35837754 0.58790464 0.84122396 0.38867075 0.17417643 0.32779485 0.35647599 0.13897743 0.79484965 0.82741109 0.65815896] Validation Epoch: 2490, F1-Score: 0.317369, mAP: 0.423296, initial mAP: 0.354492