Open JamesYang007 opened 4 years ago
Hi, I like this project! It's cool to see a project like STAN written all in C++ with the ability to add more samplers/inference methods.
I'd like to add some bullet points to this list:
Add more distributions
Add more MCMC Kernels
Add ability to trace a function of the MCMC state (i.e. logpi/Hamiltonian)
I'm looking at implementing some MCMC kernels like RWM, MALA, SM-MALA, HMC and some adaptive schemes for these (just HMC, not NUTS https://arxiv.org/pdf/1206.1901.pdf) and some others kernels which I have been working on.
I have looked at pymc3 and pyro in python but find python is very slow for reverse mode autodiff.
While I am here, is this project aiming to replicate the features of say STAN or pymc3, or focused on just MCMC say?
Hey, thanks for checking it out (and also contributing)! I'll append the list for sure - those are great points. A big goal of mine is to add a lot more kernels, some of them you already mentioned above. Feel free to put up an issue about any of them as an enhancement and take a crack at it!
We have also looked at pymc3 and indeed it was very very slow. I think it was like 40x slower than autoppl and STAN with the gaussian model benchmark example in the README. But a bigger issue was that it actually crashed on some examples (linear regression benchmark example), so it's just not as stable. I kept the pymc3 code for both examples as souvenir here - you can try running it too.
At the moment, I have no particular direction I'm thinking of with this project. But I will say that the end-goals I had in mind were 1) to allow researchers to compare accuracy and performance among different MCMC methods and 2) allow researchers to extend the library to implement their own MCMC method. While it does sound like I am focused purely on MCMC, I definitely think supporting other features in STAN and pymc3 would be attractive, so I am open to that as well. I just had the impression that posterior sampling was the main objective for those two. Did you have something in mind?
That is exactly what I had in mind! A test bed library for adding new samplers is what I would love (STAN seems like a closed system...)
Right now I am looking at adding HMC with adaptive covariance to my fork of the library.
Awesome, sounds great! Yeah, I had the same impression of STAN.
I wrote an adaptive method for NUTS which may be useful for your HMC method. It should be under mcmc/hmc. Looking forward to seeing your HMC implementation!
Even more optimizations for when variable expressions are of fixed sizenot needed based on new design