search

yuguangnudt / LBR_SPR

Deep Anomaly Discovery from Unlabeled Videos via Normality Advantage and Self-Paced Refinement (CVPR2022)
https://openaccess.thecvf.com/content/CVPR2022/html/Yu_Deep_Anomaly_Discovery_From_Unlabeled_Videos_via_Normality_Advantage_and_CVPR_2022_paper.html
5 stars 0 forks source link

Evaluation results are different #1

Closed nuclearboy95 closed 1 year ago

nuclearboy95 commented 1 year ago

Hi, thanks for sharing your work.

I followed the preprocessing instructions and ran test.py using the uploaded pretrained models. The evaluation results are quite different from the reported AUROCs as below.

The reported AUROCs are: Ped2 Avenue ShanghaiTech
test 0.957 0.928 0.721
merge 0.972 0.907 0.726
The results from my run are: Ped2 Avenue ShanghaiTech
test 0.913 0.805 0.708
merge 0.911 0.876 0.624

Can you upload the inference results (frame-wise anomaly scores) under your environment?

yuguangnudt commented 1 year ago

Hi, thanks for sharing your work.

I followed the preprocessing instructions and ran test.py using the uploaded pretrained models. The evaluation results are quite different from the reported AUROCs as below.

The reported AUROCs are: Ped2 Avenue ShanghaiTech test 0.957 0.928 0.721 merge 0.972 0.907 0.726

The results from my run are: Ped2 Avenue ShanghaiTech test 0.913 0.805 0.708 merge 0.911 0.876 0.624

Can you upload the inference results (frame-wise anomaly scores) under your environment?

Sorry, we have found that in some cases we used the wrong object detector. We have modified the codes and instructions and successfully reproduced the results. Please use the latest codes for testing. Besides, we have also trained new models from scratch and tested them, we can still achieve comparable or even better performance. We recommend training your own models and testing them so that the extracted STCs can match the models. Good luck! Furthermore, please distinguish the situations with and without motion enhancement. The reported results you mentioned are enhanced by motion (i.e. optical flow), while the results you ran out should be without motion enhancement (i.e. use_flow=False).