search

leibinghe / GAAL-based-outlier-detection

GAAL-based Outlier Detection
131 stars 58 forks source link

Calculating AUC #1

Closed KiteFlyKid closed 4 years ago

KiteFlyKid commented 4 years ago

According to the source code, AUC is derived from the 'p' (loss of discriminator) and 'y' (label of the data). 

inlier_parray = result.loc[lambda df: df.y == "nor", 'p'].values
outlier_parray = result.loc[lambda df: df.y == "out", 'p'].values

But the 'p' is to measure the similarity between the generated data and the original dataset while 'y' is to tell whether that data point in original dataset is outlier or normal. In other words, p is not the score of label y. So I think they can't be used for Wilcoxon-Mann-Witney Test to calculate AUC.

nabinabila commented 4 years ago

Hey, may I ask you , have you calculate other metrics like presicion, F1.. for this model ?

KiteFlyKid commented 4 years ago

No, I run their code and find my figures totally different from what they post in the paper. Here, I just cast doubt on the way how they calculate AUC-ROC.

nabinabila commented 4 years ago

Hey,

Thank you for tour response. For me, i am working on a school project to graduate and i m implementing GAAL model, so to evaluate the model i need to apply several metric not only the accuracy. For that, I ask if someone could help me to calculate Precision, F1 Score.. for the model !

Best

Le sam. 6 jui. 2020 à 03:17, Sean notifications@github.com a écrit :

No, I run their code and find my figures totally different from what they post in the paper. Here, I just cast doubt on the way how they calculate AUC-ROC.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/leibinghe/GAAL-based-outlier-detection/issues/1#issuecomment-639958056, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMOLTEOC2DXZ4DIPUKZA3N3RVGRJ3ANCNFSM4MHUWMSA .

-- Nabila OUNASSER

Élève ingénieur | Ecole des Sciences de l'Information P: +212 (0) 615124717 <++212+(0)+615124717> E: nabilaounasser81@gmail.com A: Casablanca Prefecture, Morocco http://www.linkedin.com/in/nabila-ounasser-70186015b https://www.facebook.com/nabila.ounasser

KiteFlyKid commented 4 years ago

Hey, I think calculating Precision, F1 is not hard stuff at all, instead just a few lines of code. Why don't u do it yourself And I wonder if you have tried PYoD. The authors have encapsulated gaal.

nabinabila commented 4 years ago

Hello,

Thanks for response. Honestly I tried to understand the logique the adopted to calculate accuracy, to have the data_y and pred_data ... i will be very grateful if you could help me doing it 🙏🏻 Best

Le sam. 6 jui. 2020 à 15:23, Sean notifications@github.com a écrit :

Hey, I think calculating Precision, F1 is not hard stuff at all, instead just a few lines of code. Why don't u do it yourself And I wonder if you have tried PYoD. The authors have encapsulated gaal.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/leibinghe/GAAL-based-outlier-detection/issues/1#issuecomment-640068711, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMOLTEP5LMSZ5NRJ6BHESDDRVJGM3ANCNFSM4MHUWMSA .

-- Nabila OUNASSER

Élève ingénieur | Ecole des Sciences de l'Information P: +212 (0) 615124717 <++212+(0)+615124717> E: nabilaounasser81@gmail.com A: Casablanca Prefecture, Morocco http://www.linkedin.com/in/nabila-ounasser-70186015b https://www.facebook.com/nabila.ounasser