search

rohban-lab / Knowledge_Distillation_AD

82 stars 46 forks source link

About the results of anomaly localization #2

Closed wangyf256 closed 3 years ago

wangyf256 commented 3 years ago

Hello, thank you for sharing. I am very interested in your work and ran this code on MVTec without modifying any settings (except those MVTec specific parameters). The results obtained are far from your paper. Maybe I haven’t noticed some details, but I don’t have any clues either. I hope you can give me some suggestions.

“——” represent my results kd

SMSD75 commented 3 years ago

Thank you for your interest in our research. We will review the code and come back to you soon.

Niousha12 commented 3 years ago

Thanks for your interest. I have reviewed the code, and unfortunately, there was a minor bug in dataloader.py file. Shuffling the test dataset will corrupt the corresponding relationship between ground truth and test data. I have fixed the problem by changing lines 110 and 130 to '' shuffle=False ". You can change it yourself or push the last commit from the repository. Also, to reproduce the result in localization you should remove the "good" folder from the "{mvtec_class_name}/test/" folder. I have also attached my config file, so you can compare it with yours.

config.zip

This issue only exists in the Github version, and the paper's results are valid.

wangyf256 commented 3 years ago

Thank you for your prompt answer, it is indeed no problem after modification.

hustCYQ commented 2 years ago

There is still a gap between the RocAUC of my test results and the results of the paper. In MVTec dataset, I find RocAUC of carpet is 0.79, RocAUC of cable is 0.89, RocAUC of grid is 0.74. The code I use is the latest version