search

HYPJUDY / Decouple-SSAD

Decoupling Localization and Classification in Single Shot Temporal Action Detection
https://arxiv.org/abs/1904.07442
MIT License
96 stars 19 forks source link

post_processing the results #14

Closed rahman-mdatiqur closed 4 years ago

rahman-mdatiqur commented 4 years ago

Hello @HYPJUDY ,

just wondering, why you are taking the max of only the scores of the positive classes and not including the background class in this line in the post_process method? Is this because, you are filtering out the low confidence detections in this line? But, the config.filter_conf_threshold is set to very small (0.10). In that case, aren't you saving a lot of potentially false positive detections in the result file?

Can you please clarify?

Thanks

HYPJUDY commented 4 years ago

You're right to some extent. The post_process function is used for finding probable "action". It's assumed the inupts are action proposals. https://github.com/HYPJUDY/Decouple-SSAD/blob/9a78cea6be27f9cc7cd512e027f00526be5ecad8/operations.py#L369-L384 Before post_process, there are two steps to filter potential backgroud proposals,

By the way, the performance of action detection evaluated by mAP could benefit from more action proposals. That's how this process works. https://github.com/HYPJUDY/Decouple-SSAD/blob/9a78cea6be27f9cc7cd512e027f00526be5ecad8/operations.py#L392 But too much proposals would be of no help, as mentioned in https://github.com/HYPJUDY/Decouple-SSAD/blob/9a78cea6be27f9cc7cd512e027f00526be5ecad8/operations.py#L410 So parameters tuning is needed for different cases. You could try to set different parameters or remove these processes to see the effect. In most cases, I adopt the same parameters as SSAD.

rahman-mdatiqur commented 4 years ago

I understand. Thank you very much for your thorough and detailed feedback as usual! I highly appreciate it.

Just One more. Though the inputs to post_process function is assumed to be action proposals, but there could be still background proposals since the below line only filters out about 60% background proposals.

 # filter len(tmpDf) from 108 to ~20~40~ 
 tmp_df = tmp_df[tmp_df.score_0 < config.filter_neg_threshold]

Therefore, just wondering, what if you take the max across all 21 classes including background inside post_process function and then save only those proposals where the confidence is maximum for positive classes? Any thoughts about this?

Thanks.

HYPJUDY commented 4 years ago

This very probably would produce worse results. If I remember correctly, the backgroud class indicating by score_0 usually have very high scores (e.g., bigger than 0.95). That's why the config.filter_neg_threshold is set to 0.98. So even if there's an action class in one proposal, its score can be smaller than background class. You can set the config.save_predict_result to True (it's by default) and see the full list predict result to get a glimpse of the scores distribution. https://github.com/HYPJUDY/Decouple-SSAD/blob/9a78cea6be27f9cc7cd512e027f00526be5ecad8/operations.py#L544-L546 I'm not sure. If you have need findings, welcome feedback : )

rahman-mdatiqur commented 4 years ago

Thanks a lot for the clarification. So, these different thresholds (e.g., config.filter_neg_threshold, config.filter_conf_threshold) are dataset dependent? In that case, what values did you use for ActivityNet?

HYPJUDY commented 4 years ago

I didn't do further experiments on ActivityNet so I don't know.

rahman-mdatiqur commented 4 years ago

you mean, you used the same thresholds for ActivityNet?

HYPJUDY commented 4 years ago

I mean, I don't know what's the proper parameters because I didn't do complete experiments on ActivityNet.

rahman-mdatiqur commented 4 years ago

Ok, I understand. Thanks again for your nice feedback.