Results are not reproducible at all

reml-lab / mTAN

Code for "Multi-Time Attention Networks for Irregularly Sampled Time Series", ICLR 2021.

MIT License

111 stars 22 forks source link

Results are not reproducible at all #2

Open CubicQubit opened 2 years ago

CubicQubit commented 2 years ago

I was able able to run the code perfectly but the hyperparameters given for mTAN-full do not achieve 0.858 AUROC on physionet2012. The highest is AUROC ~0.827 with AUPRC ~0.45.

This is achieved with this command: python3 tan_classification.py --alpha 100 --niters 300 --lr 0.0001 --batch-size 50 --rec-hidden 256 --gen-hidden 50 --latent-dim 20 --enc mtan_rnn --dec mtan_rnn --n 8000 --quantization 0.016 --save 1 --classif --norm --kl --learn-emb --k-iwae 1 --dataset physionet

satyanshukla commented 2 years ago

Could you please post the training logs here?

CodeNinjaja commented 2 months ago

Could you please post the training logs here?

I also encounter the question. I've attached the training log below. mtan.log

I am looking forward to hearing back from you.