Alternative tutorial on implementing samplers

[torfjelde]

I did a talk on Turing.jl for a statistical audience earlier today and described how it's possible to implement a samplers in AbstratMCMC.jl in such a way that it's very easy to use the resulting sampler to target both Turing.jl and Stan models (through BridgeStan).

Afterwards I was asked if there was some tutorial or something online outlining how to do this in a detailed way. After having a look at the current docs on how to use the externalsampler, I felt they were a bit overly complicated and didn't quite get all the points across, and so I did a rewrite of it.

I think this version will be very, very useful for inference researchers who want their methods to become available for a larger userbase:)

EDIT: Note that I haven't added example with StanLogDensityProblems.jl, but I think we maybe should, as it would be quite nice.

I'm also thinking we should make the example a bit more complicated later in the tutorial, e.g. adding stuff like "how to test the sampler with MCMCTesting.jl" from @Red-Portal , adding a few more things to the MALAState, e.g. isaccept to indicate whether it got accepted, maybe add some adaptation, etc. But I think the current version should be merged first, and then we can build on it later :+1:

[github-actions[bot]]

Preview the changes: https://turinglang.org/docs/pr-previews/468, Please avoid using the search feature and navigation bar in PR previews!

[yebai]

Let’s add this as a new docs instead of overriding an existing one. I am aware of users relying on the pathfinder and external samplers in current docs.

[torfjelde]

But the current docs are somewhat out of date + it's too similar in what they describe IMO. Maybe it makes more sense to add another section at the end of this tutorial demonstrating some examples, where I'll add back the examples in the original?

[torfjelde]

Maybe it makes more sense to add another section at the end of this tutorial demonstrating some examples, where I'll add back the examples in the original?

Happy with this Hong?

[yebai]

Let's keep both tutorials. This tutorial can be called something else, but go to the For developers (Inference) section.

The previous tutorial on external samplers provides the basic mechanism for calling AbstractMCMC-compatible external samplers. This PR can build upon that and provide instructions on implementing a new AbstractMCMC sampler.

It's okay to reorganise docs tutorials later, but I felt we needed to do that after making a plan and reviewing all existing docs.

[torfjelde]

Done :+1:

[torfjelde]

@yebai notice that I've implemented a version of MALA which uses a single leapfrog step rather than the "standard" formulation of MALA. My original intention was to add a final section on a more "complex" version being autoMALA. Buuut I'm uncertain if this is worth it or not. Should I still pursue this or just leave it for now?

[yebai]

@mhauru can you take a look at this tutorial?

[torfjelde]

This should be ready now:)