Different evaluation strategy

[LiUzHiAn]

Is your evaluation strategy the same as the official?

I notice you average the AUCs of 140 testing abnormal videos. If I didn't remember wrong, the official calculated the whole videos at the same time.

[LiUzHiAn]

Besides, the testing normal videos seem not to be used.

[seominseok0429]

you're right. I only used 140 because I wanted to calculate AUC using sklearn. If all the remaining 10 normal videos are calculated, the AUC is 86. The code can be easily modified.

[LiUzHiAn]

Have you ever tried to evaluate the AUC of all the videos at once?

To be more specific, suppose we have 3 test videos with 1000 frames each. And the first video is normal, the rest two are abnormal ones. You should contact the gt_labels and predicted scores of all three videos and use sklearn to calculate the AUC. What is the result then?

Thx.

[Karthikeya-t]

Can you please tell me How to add evaluate this model , Auc curve code of this model

Thanks for helping

[wjtan99]

@LiUzHiAn I tried your way, and the AUC is not identical but about the same.
@seominseok0429 What training parameters did you use? I got AUC=0.82, but 0.845. Thanks.

[seominseok0429]

Ours full code available at CODE