search

tianyu0207 / RTFM

Official code for 'Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learning' [ICCV 2021]
324 stars 77 forks source link

About epoch and AUC #24

Open Yii-maker opened 3 years ago

Yii-maker commented 3 years ago

Hello, Thank you for your excellence job！ 1.May I ask why you wrote in the article that epoch is 50, but in the code it is 15000? 2.When I reproduced the data set of Shanghai, I found that the AUC was not very high. May I ask if you have adopted the learning rate attenuation mechanism?Or anything else you forgot to upload to the code?

KhalilBergaoui commented 3 years ago

@Yii-maker I also tried to run the code on Shanghai pretrained features and I got ~89% AUC . Did you obtain the same score ?

tianyu0207 commented 3 years ago

@Yii-maker I also tried to run the code on Shanghai pretrained features and I got ~89% AUC . Did you obtain the same scor

tianyu0207 commented 3 years ago

Hi. The checkpoint is selected based on the highest AUC, following the previous anomaly detection papers. I did not apply any mechanism other than the code in this repository. With I3D you should get around 96.XX - 97.XX based on different random seeds. With C3D, you should get around 90.XX - 91.XX AUC.

KhalilBergaoui commented 3 years ago

Hi @tianyu0207 thank you for your reply. I rerun your code with different random seeds (using your I3D features) but test AUC always converges around 90%. I tried to visualize the training loss and i noticed the total loss (cost term in the code) increases after some epochs. I tried to decompose the loss terms and it seems to me that the increase is due to the margin loss (um_loss term in the code) (in the plot I tried a few configurations with the default being yours implemented in the code, and only for visualization purpose I did not multiply by alpha) :

total_loss margin_loss

Did you encounter the same problem ? If so how did you manage to get a decreasing training loss ? Do I need to change any of the hyperparameters to get around 97% AUC ? Thanks !

kapildeshpande commented 2 years ago

@KhalilBergaoui Have you got 96-97% AUC, I have tried the same code using I3D features given in the Readme but got only 92% AUC.

botao-cao commented 2 years ago

@KhalilBergaoui Have you got 96-97% AUC, I have tried the same code using I3D features given in the Readme but got only 92% AUC. I got 94% AUC and I can't get 96-97% AUC.

albertxwz commented 1 year ago

@KhalilBergaoui Have you got 96-97% AUC, I have tried the same code using I3D features given in the Readme but got only 92% AUC. I got 94% AUC and I can't get 96-97% AUC.

I got around 95.7% AUC using I3D features of SHT.

epoch: 4575
0.956566358955364

And I can't get AUC above 97%.