Closed GinGinWang closed 5 years ago
Hi Gin,
from the supplementary material of the paper (chapter 7), one can infer lambda = 1 batch size = 256 learning rate = 10^-4 Optimizer = adam They stopped their training after a fixed number of 200 epochs (see closed issue) and monitored the LSA-loss (reconstruction loss plus autoregression loss (neg. llk)). However, if I train with these parameters, I only achieve an average AUROC-NS of 0.817 and 0.916 with a batch size of 64. Thus, they probably change some of these parameters for a better convergence. With batch size 256 it does not converge well but maybe I made a mistake. I attached my code for training (I changed the val() function in the mnist.py code to extract the first 90 % of the training data instead of the last 10 %). train.py.zip
Hi and thank you for your interest.
@GinGinWang as correctly pointed out by @mathue, training details are provided in the supplementary material.
@mathue at a first glance, I can see a major issue in your training code:
val
function in the dataset implements the validation split, which is performed by shuffling indexes and taking the last 10%. You should implement a train
function that puts the dataset into training mode (by taking the first 90% of training indexes).Hope this helps, D
Hi Davide,
thanks alot for your reply. You are right, that's a shame, I made a mistake and trained on the 10 % validation set. When training on the 90 % training set for 200 epochs, I also achieved 0.972 AUROC-NS which is very close to what you reported in the paper.
Best regards Martin.
Hi Davide, Sorry to bother you again. I replicate your experiments on cifar10 using all the parameters you recommended in your paper. After 200 epochs, I got AUROC 0.6. I also tried more epochs (1000 epochs), the AUROC did not increase. But I could have good results on the mnist dataset ... Is there any other training trick that is not mentioned in your paper? Thanks a lot for your help!
Hi, I am replicating your experiment's results. Could you give more the following details about the training on mnist and cifar10 which will replicate the results on your paper?
Thanks a lot! Best, Gin