add advanced mc

[mswwilson]

Not urgent to merge soon. I just wanted to get these old codes back in before they disappear.

The new files in examples/ do not need to stay - they are just to show the usage

[kbarros]

This looks good to me, thanks Matt. I wonder if we can find more descriptive names, since these will be going into the public API. A google search of "GenEnsemble" does not turn up much. My first thought is WangLandau and ParallelWangLandau, which work well with google search. Is there something we lose? (i.e. is there something special in MUCA that's not captured by the above names?) cc @yingwaili .

I also wonder how we should organize the examples directory -- maybe these should go in a subdirectory examples/advanced_sampling ?

[mswwilson]

Thanks. Yeah, I struggled with what to name these types. GenEnsemble stands for 'Generalized Ensemble', which may be too obscure. These codes can be used to perform either Wang-Landau or multicanonical sampling. Wang-Landau converges towards a multicanonical ensemble in late iterations, but I know the (small) community can be picky about the distinction because multicanonical sampling actually satisfies detailed balance.

I originally had them as 'Multicanonical' and 'ParallelMulticanonical', and also considered something like 'FlatHistogramSampler', although the histogram criterion doesn't have to be chosen as flat. It may be more appropriate to call Wang-Landau a type of Multicanonical sampling rather than the other way around, but I'm open to any alternatives.

[kbarros]

Ying Wai is in the process of preparing review. One thing we noticed is that Wang-Landau requires a proposal function, rather than a general sampler. It would be nice to use the latter instead, e.g., to enable the use of global moves (like a Langevin step) within Wang-Landau. Unfortunately, the step! function for Langevin provides no facility to "undo" a move. To enable this, we would need to store the trial update in a separate state array. This, in turn, would require creating some new data structures. I created https://github.com/SunnySuite/Sunny.jl/issues/76 to discuss this.

[kbarros]

@ddahlbom this PR introduces three new examples. Where do you think they should live? I propose that the top level of the examples/ directory be only the files that appear in the docs through Literate.jl. I.e., these would be highly polished files that we want people to see first. We already have a subdirectory examples/longer_examples. Maybe we should rename it examples/extra and put all extra examples in there?

For a future PR, I think a demonstration of these new MCMC algorithms can be migrated into, perhaps, the Ising2D example.

[ddahlbom]

An extra subdirectory sounds like a good solution. I agree the top level is just for Literate files that go on the docs. (Probably we should change longer_examples to something more explicit, since this directory is exclusively for the literate files that are used to make notebooks for the SunnyTutorials repo. Maybe longer_examples -> sunny_tutorials.)

In any case, it's exciting to see all this MC functionality going into Sunny!

[kbarros]

OK, so we would have:

examples/
     *.jl   #  polished Literate files, users should look at them first
     longer_tutorials/*.jl  # used to generate polished notebooks for SunnyTutorials
     extra/*.jl  # miscellaneous stuff can land here if it's not polished yet

A lot of the Monte Carlo design is still in flux, and the new examples Matt made may be using this unpublished API.

[kbarros]

There is high demand for these features, so I'd like to get this merged. Remaining tasks:

• Rename GenEnsemble -> WangLandau
• Rename ParallelGenEnsemble -> ParallelWangLandau
• Move the new examples into examples/extra/

Other polish can be added later. @yingwaili do you agree?

[mswwilson]

Ok, I made those changes. I'm going out for a bit, but can add a doc string to sample! later

[kbarros]

I have merged this. We can address Ying Wai's feedback in follow up work.

[yingwaili]

Regarding the MC examples: once they are polished, I would suggest creating a new directory dedicated to Monte Carlo methods (e.g. called MonteCarlo instead of extra) and move the examples there?

[kbarros]

This is a good idea. The current examples/extra files are really more like tests. This MCMC code will need to be further polished, and then we will make a proper example that can be included in the main docs. Thanks for your input!