search

gathierry / FastFlow

Apache License 2.0
132 stars 24 forks source link

Output interpretation #8

Closed RoelofKuipers closed 2 years ago

RoelofKuipers commented 2 years ago

Hello Gathierry,

I am a bit puzzled on how to interpet the output of the evaluation. In line 97-98 of main.py in the eval_once function, I get negative results as my output. It does give me a proper ROC_AUC value. How do I translate this to boolean values?

Imagine I want to add another image of a hazelnut and see if it has an anomaly or not, how would I interpret the results of the model?

Best, Roelof

RoelofKuipers commented 2 years ago

This is my first outputs (from outputs = ret["anomaly_map"].cpu().detach())

Screenshot 2022-05-17 at 14 45 10

gathierry commented 2 years ago

Hi @RoelofKuipers , as the output of fastflow is the likelihood (greater value = more normal), I convert it to negative to represent the anomaly score, that's why the output is negative. https://github.com/gathierry/FastFlow/blob/d275b79d47d6e272115d45fd7fc0f29cca0f5107/fastflow.py#L151

To translate it to boolean values, you need to set a threshold. then outputs > threshold is a boolean array in which True -> abnormal, False -> normal

Setting a threshold is kind of tricky, it is usually determined based on the ROC curve. And I believe you'll need different threshold values for different datasets.

RoelofKuipers commented 2 years ago

Thank you @gathierry , makes total sense. I simply got confused with the -prob as a likelihood for a while. But this doesn't matter at all of course.