search

sming256 / OpenTAD

OpenTAD is an open-source temporal action detection (TAD) toolbox based on PyTorch.
Apache License 2.0
156 stars 10 forks source link

Proposal Generation Only #17

Closed Echo0125 closed 4 months ago

Echo0125 commented 4 months ago

Hello, I am interested in performing only the task of Temporal Action Localization and do not want to classify each proposal. Could you please guide me on how to modify the code accordingly?

sming256 commented 4 months ago

The proposal generation is supported in the next version of OpenTAD, which should be released next week. We will provide BMN/GTAD as an example for temporal action proposal generation task. Stay tuned!

sming256 commented 4 months ago

Sorry, I just checked. We have already released the proposal generation code in the current version. Please check here as an example.

Echo0125 commented 4 months ago

If I want to use actionformer config for e2e-tal without classification, which part of the config do I need to modify? Just modify the rpn_head in base? Thanks!

sming256 commented 4 months ago

On ActivityNet, ActionFormer sets the num_classes=1 and uses an external classifier to do classification. If you want to measure their AUC performance, you only need to change the evaluation metric.

evaluation = dict(
    type="Recall",
    subset="validation",
    topk=[1, 5, 10, 100],
    max_avg_nr_proposals=100,
    tiou_thresholds=[0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95],
)

On THUMOS, you need to change the num_classes to 1 and modify the annotation to have 1 class.

However, since ActionFormer didn't report their Recall results, so I am not sure whether it is better than BMN or not. If you have more questions about this or need more discussions, please send an email to me at shuming.liu@kaust.edu.sa.

Echo0125 commented 4 months ago

Thanks for your reply! I'll give it a try.