search

oopil / PSAD_logical_anomaly_detection

25 stars 3 forks source link

Why don't use Pixel-level AUROC? #2

Open hxx-who opened 3 months ago

hxx-who commented 3 months ago

Our proposed PSAD and other comparison methods are evaluated using image AUROC scores. As there is no ground truth for segmentation task. However, I see mvtec_loco_ad provides the segmentation mask. I'm wondering why you don't evaluate using pixel AUROC scores?

oopil commented 2 months ago

I mean that measuring segmentation performance is not possible because there should be pixel-level label on the different components. I am aware of the binary map you mention but our model is not evaluated on the localization capability in our AAAI paper. For now, our model predict anomaly score directly. It could be extended in our future work. The map contain all possible anomalous area that it could be too large. For this reason, the paper that proposed GCAD uses a different metric sPRO to measure the localization capability.