search

caoyunkang / Segment-Any-Anomaly

Official implementation of "Segment Any Anomaly without Training via Hybrid Prompt Regularization (SAA+)".
720 stars 75 forks source link

About Evaluation Metrics #35

Open Myzhao1999 opened 3 months ago

Myzhao1999 commented 3 months ago

Hi~ Thanks for your interesting work and the open code. I noticed that you mentioned using max-F1-pixel and max-F1 region as Evaluation Metrics in the paper. Would you please tell me if you have tried to use image-level AUC and pxiel-level AUC to evaluate this work like the previous work?

caoyunkang commented 1 month ago

Thanks for your question! SAA+ is not good at AUROCs. Intuitively, higher AUROCs require continuous anomaly scores, but the scores produced by SAA+ are typically small or large, which makes SAA+ weak in AUROCs. In contrast, max-F1 can better reflect the binary segmentation performance, which can better show SAA+' superiority.