Closed zzzhezeng closed 2 years ago
Ugh, yeah it looks like this was on our end (see also the longer email) and also luckily only yacht rather than any other datasets. I spent a while re-tuning yacht and it looks like it was only the weight decay that was off. Switching to a lower weight decay of 1e-3 provides a more similar LL and RMSE result:
edit: i'll fix the script in the code tomorrow.
Hello,
The approach and results presented in the paper "Subspace Inference for Bayesian Deep Learning" impress me a lot and I would like to reproduce the experiments, specifically the UCI experiments, using this repository.
I followed the instructions and process described in the
README
and managed to get things run. However, I failed to obtain similar numbers for certain datasets.To be more precise:
versions of libraries match.
the datasets are downloaded from Google Drive as suggested in the
README
underdrbayes/experiments/uci_exps
.all experiments on small UCI datasets are launched by the script
run_ucismall.sh
underdrbayes/experiments/uci_exps/bayesian_benchmarks/tasks
.the reproduced unnormalized test likelihoods for the
yacht
dataset are significantly different from those reported in the paper as shown below.I would be extremely grateful if anyone could suggest what I could've done wrong in my reproduction experiments.
Thank you in advance!