search

Julian-Wyatt / AnoDDPM

CVPR Workshop paper - AnoDDPM: Anomaly Detection with Denoising Diffusion Probabilistic Models using Simplex Noise
https://julianwyatt.co.uk/anoddpm
MIT License
174 stars 28 forks source link

About the evaluation. #5

Closed 0Godness closed 2 years ago

0Godness commented 2 years ago

Could you please provide the evaluation code for MVTEC dataset or other image dataset to reproduce the AUC results in your paper? Thanks a lot!!

Julian-Wyatt commented 2 years ago

Hi,

The MVTec dataset results shown are preliminary; therefore, the AUC results solely cover the MRI dataset.

There shouldn't be much refactoring to perform the AUC analysis on an alternative dataset if you so wish.

Let me know if this helps

0Godness commented 2 years ago

But I want to know how to calculate AUC results in your code for any dataset. Could you please provide a simple example? Thanks!

kbqn20 commented 2 years ago

Please, how can I get the requirements.txt. file？

Julian-Wyatt commented 2 years ago

But I want to know how to calculate AUC results in your code for any dataset. Could you please provide a simple example? Thanks!

Hi, just to double check, you're unsure with how to perform ROC-AUC calculations on any dataset?

I feel this is open to how you would like to implement your other datasets. For example, if you would like to analyse the performance of MVTec, which is 1024x1024, you may have to incorporate a sliding window, to analyse each section and then concatenate all of the true positive rate (tpr) and false positive rate (fpr) values; or alternatively, you can resize the image, calculate the tpr and fpr values, and concatenate them across the whole dataset where you would then calculate the ROC for the whole dataset

This may help as a small example: https://stats.stackexchange.com/questions/538842/about-roc-curve-in-segmentation-model

0Godness commented 2 years ago

Hi, I am sorry that I make some mistakes about your metric code. Now there are no problems! Thanks again for your kind and fast reply and the project is very nice!

zideliu commented 2 years ago

Could you please provide the evaluation code for MVTEC dataset or other image dataset to reproduce the AUC results in your paper? Thanks a lot!!

Do you know how to use detection.py? I want to get the result image of detection through this file, but I don't know how to get this image I would really appreciate it if I could get a reply