Implementing and correcting the base set of swap strategies, plus wrapping swap moves as part of steps

[HarrisonWilde]

Contributions:

• Merged the swap move part of a step with the step itself:
  • Rather than having some steps returned by AbstractMCMC.sample that correspond only to the result of swapping temperatures between chains, we carry out the check for whether we should_swap and do the swap itself as part of the AbstractMCMC.step and always progress the chains using the underlying exploratory samplers / kernels as well, this leads to a 1:1 correspondence between nsamples specified at sample time and the total number of steps taken by the samplers, which is desirable.
• Implementations of standard swap strategies:
  • Provide correct implementations of SingleRandomSwap, RandomSwap, ReversibleSwap and NonReversibleSwap in line with their definitions in the referenced literature. Reversible and Non-Reversible schemes are particularly important for reasonable baseline performance of a functional PT package, as they increase our robustness to untuned temperature ladders.
  • Renamed StandardSwap to SingleSwap for clarity.
  • Added NoSwap for debugging / disabling of swap functionality.
• Cleaned up swapping.jl:
  • Generalised to accept i and j inverse temperature indices to do a swap between.
  • Removed unnecessary arguments from swap_attempt (we weren't using the total_steps functions in adaptation anyway, unless I am missing something?)
  • Clearer variable names and references
• Adaptation is not supported for RandomSwap and SingleRandomSwap, due to them involving swaps between non-neighbouring chains. Worth noting it doesnt seem to work well in general currently. Have added a check and raise for the situation where user tries to adapt under these strategies.

Pending TODO:

• Could further clean up swapping.jl by renaming the variables ending in _other, potentially remove the versions of functions that take many args given we are explicitly supporting LogDensityProblems as a base for most of we want to do for now, could add back more complex signatures for things like compute_tempered_logdensities if required down the line.

[torfjelde]

Merged the swap move part of a step with the step itself:

I'm confused; where is this happening? I'll have a closer look later; might have just missed it on the first viewing.

[HarrisonWilde]

LGTM! Just need to make sure tests are passing. You all good locally? I'll set up Github actions with CI in a PR now.

All passing on my end other than the final MH one for the Turing model