About performence

[nqxjzm]

Great work！ I would like to know what specific changes you have made to make thumos14 perform better than before

[tzzcl]

In Table 3, we presented an detailed ablation on model design. Table 3 shows how we can reach 62.6% mAP on THUMOS14, compared to other methods. Moreover, we have around 67 mAP now.

[nqxjzm]

I'm sorry I didn't express what I meant clearly, what I want to ask is what specific changes have you made to increase the performance to 67 mAP compared to the previous performance.

[nqxjzm]

Because I can't reproduce your latest results

[tzzcl]

For the latest results, you may want to refer to the README and this commit for details.

[happyharrycn]

The commit is this one. Previously, one location on the feature pyramid was matched to a single ground-truth action during training. This creates an issue on THUMOS14, where there can be two actions with the same onset and offset yet with different labels. You can also see the discussion in this issue #10.

Using different machines and software decks will produce a minor variation in the results, usually in the range of 1 mAP. Let me know if you could not reproduce the results up to this range.

[nqxjzm]

Thanks for your detailed reply！I use your current code,but I just get 65mAP at tiou=0.5 on thumos14 as follows |tIoU = 0.30: mAP = 75.47 (%) |tIoU = 0.40: mAP = 72.61 (%) |tIoU = 0.50: mAP = 64.98 (%) |tIoU = 0.60: mAP = 55.22 (%) |tIoU = 0.70: mAP = 41.51 (%) Avearge mAP: 61.96 (%) All done! Total time: 39.79 sec

Process finished with exit code 0

[tzzcl]

The performance is similar to what we reported before in the report (65.6 vs 65.0 mAP with tIoU 0.5) and (62.6 vs 62.0 average mAP). I've run several times with the latest code on THUMOS14, the mean mAP is close to 66.0 mAP. Please make sure your code is at the latest commit 5d8df9f and you use our features.

[happyharrycn]

I second to Chenlin's comment. By a quick look at your numbers, I think you are either using an older version of the code, or an older version of the config file. To test this, you can check the terminal output for training or inference, where the config will be printed at the very beginning. Do you have test_cfg -> multiclass_nms set to False? If so, you are on an older branch.

[nqxjzm]

Thanks for your reply very much! I got the latest performence. Amazing work! |tIoU = 0.30: mAP = 81.72 (%) |tIoU = 0.40: mAP = 78.05 (%) |tIoU = 0.50: mAP = 70.87 (%) |tIoU = 0.60: mAP = 59.42 (%) |tIoU = 0.70: mAP = 43.73 (%) Avearge mAP: 66.76 (%) All done! Total time: 31.43 sec