Are the baseline models and HMC sample checkpoint available?

[timxzz]

Hi Pavel!

I saw that on the competition website, it says

... we will release several baselines, including stochastic MCMC, variational methods, and deep ensembles. We will upload HMC samples on other model-dataset pairs over time.

but I can only find the HMC checkpoints on CIFAR-10, 100 with PreResNet-20 at the Colab link.

I am wondering if you can point me to the code base for the baseline methods, and other available HMC checkpoints? Thanks a lot!!

Best, Tim

[izmailovpavel]

Hi @timxzz,

The evaluation phase HMC samples are available here: https://github.com/izmailovpavel/neurips_bdl_starter_kit/tree/main/eval-phase.

Please also see the discussion here: https://github.com/izmailovpavel/neurips_bdl_starter_kit/issues/4.

For the baselines, we have a few methods implemented in JAX here: https://github.com/google-research/google-research/tree/master/bnn_hmc.

[timxzz]

Hi @izmailovpavel,

Thanks for your reply and the pointer to the baselines! At the link you mentioned above, I can only find the predictions from HMC samples, but not the HMC model checkpoints -

The evaluation phase HMC samples are available here: https://github.com/izmailovpavel/neurips_bdl_starter_kit/tree/main/eval-phase.

I am wondering if you still have the model checkpoints for those samples.

Best, Tim

[pavel-izmailov]

Hi Tim, we unfortunately only released the sample checkpoints for the CIFAR-10, CIFAR-100 and IMDB. I do not have access to the other checkpoints and cannot release them. Sorry about that.

[timxzz]

I see. Thanks anyway for your help!