search

AlessioGalluccio / FastFlow

an implementation of the architecture of FastFlow (Jiawei Yu et al.)
MIT License
40 stars 13 forks source link

About MVTec dataset accuracy #8

Open xfby2016 opened 2 years ago

xfby2016 commented 2 years ago

Hello, have you ever tested the mvtec dataset accuracy? I tested the auroc of the carpet category with only 0.49，It is very different from the paper. Is there any guidance and suggestions ? Thanks!

AlessioGalluccio commented 2 years ago

Hello @xfby2016, I haven't tested the mvtec dataset yet. I'm still trying to recreate the model of the paper. For example, I still have issues in understanding which type of loss the paper used, since it is not clearly specified. However, thank you for testing. I hope it will improve when I will solve the mystery behind the loss function. If you get some clue, let me know. Best, Alessio

letmejoin commented 2 years ago

Hello @xfby2016, I haven't tested the mvtec dataset yet. I'm still trying to recreate the model of the paper. For example, I still have issues in understanding which type of loss the paper used, since it is not clearly specified. However, thank you for testing. I hope it will improve when I will solve the mystery behind the loss function. If you get some clue, let me know. Best, Alessio

Hi AlessioGalluccio, I find the loss used now maybe not match with the paper. And the author is not specified in the paper, otherwise said "adjust the loss function accordingly". I test several class of MVTec, the accuracy is not good, even bad on some case. And the loss used now could be very large nagetive number, larger than 20000! So, have you understand the loss now?

AlessioGalluccio commented 2 years ago

I haven't understood it yet. I'm open to suggestions