Closed DingShizhe closed 2 years ago
The result of the script is the mean prediction error of the whole instance, and the mean prediction error of interpolation points is presented in the article. I've reproduced it.
The result of the script is the mean prediction error of the whole instance, and the mean prediction error of interpolation points is presented in the article. I've reproduced it.
Thank you for sharing your experience and findings! I've been working on reproducing the results as well and have encountered similar challenges. However, my results (around 3.7 ^ 10-3) are still higher than the reported results. I mainly changed the evaluation metrics to only consider output the MSE for interpolated points.
Did you find any subtle implementation details that significantly affected the results? For example, does the decoder part also only see the subsampled_tp? If possible, can you share your code snippet for the training loop?
Thank you for your help in advance!
The MSE score of interpolation performance on PhysioNet with 90% data points observed is 4.798 ± 0.036 ×10^−3. But I get 1.2 ×10^−3 after I run
python3 tan_interpolation.py --niters 500 --lr 0.001 --batch-size 32 --rec-hidden 64 --latent-dim 16 --quantization 0.016 --enc mtan_rnn --dec mtan_rnn --n 8000 --gen-hidden 50 --save 1 --k-iwae 5 --std 0.01 --norm --learn-emb --kl --seed 0 --num-ref-points 64 --dataset physionet --sample-tp 0.9
.If the test data and prediction results are restored to the original scale according to the normalization of data preprocessing, then I get MSE of 4225.6895.
So how can I get the results of the paper presented?