tweag / chainsail

Replica Exchange sampling as-a-service
MIT License
11 stars 1 forks source link

Script for Chainsail walkthrough video #366

Closed simeoncarstens closed 1 year ago

simeoncarstens commented 2 years ago

To make Chainsail more accessible to potential users without needing them to actually try it out, we agreed on producing a walkthrough video. Let's discuss the script for that video in this issue.

Here's a first suggestion:

  1. Quick introduction of the speaker, Tweag, and a pointer to the MCMC blog post series
  2. Super brief discussion of multimodal probability distributions:
    • where do they occur in nature
    • show example (eog bimodal.png)
      • label switching, e.g. clustering, Bayesian neural networks
      • latent variable models, e.g., probabilistic PCA
      • Bayesian hierarchical models
      • ambiguous data (biomolecular structure determination)
    • what are current issues with sampling them
    • how does Chainsail help
      • implement Replica Exchange with automated parameter tuning
  3. Demonstration of failing sampling on 2D Gaussian mixture example. This offers also the possibility for a shameless Nix plug
  4. Super brief explanation of general Chainsail workflow:
    • write probability distribution
    • start job on website
  5. Explain chainsail-helpers PDF interface
  6. Zip & upload PDF:
    $ zip mixture.zip probability.py
  7. Start job on Chainsail website, with short rundown of most important parameters
  8. Wait for first plots to show up, explain dashboard
  9. Fast-forward until job is finished
  10. Download results, run postprocessing script (from chainsail-helpers)
    $ mkdir downloaded_results
    $ cd downloaded_results
    $ cp ~/Downloads/results.zip ./
    $ unzip results.zip
    $ concatenate-samples production_run chainsail_samples.npy
  11. Demonstrate difference in sampling using comparison script
    $ cd ..
    $ python compare.py sc_samples.npy downloaded_results/chainsail_samples.npy
  12. Finish up with call to action, support@chainsail.io email address, links to blog posts and pointer to @Etjean's soft k-means example (video / notebook)

I will produce a (unpolished) test video of this on the basis of which we can then discuss how well this script works and then likely amend it.

Etjean commented 2 years ago
  1. Super brief discussion of multimodal probability distributions:
    • where do they occur in nature
      • mismatch between likelihhod and prior

Are you sure about this one? IIRC we said in the last Chainsail meeting that since it's the product likelihood*prior, and not the sum likelihood+prior, that should not produce a multimodal.

Other than that, I think the script is great and feels natural.

simeoncarstens commented 1 year ago

Walkthrough video has been ready since the August 2022 release and is available here.