Advanced examples, we need to pick some of these and implement them.

[franciscovargas]

• Bayesian Neural Network on MNIST (maybe CIFAR). Pick the gold standard architecture for the HMC model here https://arxiv.org/pdf/2102.06559.pdf.
• Bayesian ICA from the SGLD paper https://www.stats.ox.ac.uk/~teh/research/compstats/WelTeh2011a.pdf (MEG dataset)
• A good Bayesian GMM example (Find a reference from some VI paper)
• What other Bayesian model do practioners care about and use VI for ? Maybe Robust regression ?
• If we had a lot of time we could do a topic model and collapse out the discrete variables
• How do we handle latent variables which are distinct for each data point ? like a VAE , how do we amortise the drift ? By amortising we mean like in VAEs I mean something like \phi_i = NN(x_i) type of thing where \phi_i is the variational parameter for z_i . Its not super clear how this would be done on the drift. It might be possible to do something with Ito's lemma evolve some form of x_i dependant reparametrisation of the SDE instead.
• At the same time all of these x_i are IID so it might just be able to similare the same SDE N times if we decouple things with the log appropiately. This only applies when the z_is are not correlated but the math works out if you assume that optimal drift should depend only on z_i for the ith dimension then this works out. I think it shouldnt be too hard to prove that this is true either. This is a tough example despite being an easier problem than bayesian inference , but it would look very good.

[franciscovargas]

On a second note I think the final one is not so hard and can be done ina heuristicy fashion where each drift dimension is actually b_i(x_i, z_i(t),t) the same network but x_i amortises it this is quite cool and easy and I think the independance in the drift proof is actually quite trivial if we use results from huang et all , so doing a VAE is in scope.