How to get the action labels?

[shakib-root]

Currently when I infer or evaluate a video with the pretrained ActivityNet model, the action label is always 0[time segments are available as expected]. The reason I guess is the pretrained model was trained with the setting num_classes=1

Also, the valid_one_epoch uses external score file to get the action labels. How was the score file generated and how to get the action label if one does not have such file? Is there any parameter in the config file that needs to be changed?

[tzzcl]

For the external score on ActivityNet, we use the score from CUHK & ETHZ & SIAT Submission to ActivityNet challenge 2017. If you do not have the external score file for your dataset, you may just disable the score fusion part. Please refer to our THUMOS14 config for more details (L50).

[shakib-root]

Thanks for the explanation of external scores. I tried using the pretrained Activitynet model following the config you mentioned and setting the multiclass_nms: True and ran inference/test(topk=5) with a kickboxing video from activitynet dataset. The output is shown below:

{'video-id': ['doing_kickboxing', 'doing_kickboxing', 'doing_kickboxing', 'doing_kickboxing', 'doing_kickboxing'],
't-start': array([3.5618675e-01, 5.6287831e-01, 3.0202144e-01, 2.3099737e+01, 2.7876747e+01], dtype=float32),
't-end': array([29.75667  , 13.629519 , 29.829634 , 29.884605 , 29.819345 ], dtype=float32),
'label': array([0, 0, 0, 0, 0]),
'score': array([0.74504584, 0.1019143 , 0.0653148 , 0.05246412, 0.02856769], dtype=float32)}

Here, the action-labels are still 0. Am I missing something?

[tzzcl]

Hi, The ActivityNet pre-trained model is trained with one class. Directly changing to multi-nms won't make any changes here. You may need to re-train the ActivityNet model with multiclasses.

[tzzcl]

Closed due to inactivity.