search

microsoft / anomalydetector

SR-CNN
MIT License
257 stars 70 forks source link

Reproducibility Yahoo results #33

Open chrisby opened 3 years ago

chrisby commented 3 years ago

Hello,

was anyone able to reproduce the Yahoo results from the paper? The results I am able to produce are not even close to the reported performance?

@Authors, it would be great if you could provide the respective hyperparameter settings.

All the best, Christian

yazhubaixue commented 3 years ago

same problem on KPI dataset, could anyone able to reproduce the results share the hyperparameter settings?

Fayeben commented 3 years ago

same problem on KPI dataset, could anyone able to reproduce the results share the hyperparameter settings?

XxidroxX commented 3 years ago

News about the parameters?

StCross commented 2 years ago

Any score of your reproduction? Mine are quite low (f1=0.22) and I dont know why. I set dely=7, thres=0.495

mertcanboz commented 2 years ago

I also could not reproduce the results on KPI dataset. Dear authors, could you please share the hyperparameters? Also for the 'coldstart' scenarios, what are baseline models that you stated in the paper that you have used? I failed to find them.

@guinao @amazonnnn @conhua @congruihuang @likebupt @sccc19

oliveshell commented 2 years ago

Me too in the kpi dataset, looking forward the response please.

emerrf commented 2 years ago

I am not the authors but I managed to reproduce the results for KPI dataset. I implemented a quick parameter search for SR approach and found slighlty better results than the paper (F1 in test: 0.675). However, the SRCNN was more challenging and managed to reach (F1 in test: 0.749).

I forked the repo and change it quite a bit to make it run on any machine. Please have a look at: https://github.com/emerrf/anomalydetector If you have any questions or wants to discuss my changes, feel free to open an issue there.

wwulfric commented 1 year ago

@emerrf thanks for your excellent work. It runs easier.