mini tutorial for docs

[orbeckst]

The docs should start with a small tutorial that shows how to use GromacsWrapper.

This is also important for the paper #147 because they will review docs together with installability etc.

tutorial

• installation with Gromacs (can use conda) so that it is completely self contained
• detection and configuration (or rather, how little need there is to configure anything)
• can use the AdK tutorial as a starting point: condense and show
  1. how to use GW explicitly,
  2. how to use gromacs.setup to streamline the process.

[richardjgowers]

one of the places tests are weakest currently is cookbook, any many of those "recipes" I don't understand because they're domain specific. So if someone could also expand the recipes into regression tests they would win the favour of the coverage gods.

[orbeckst]

Some of the recipes are arcane... and it might be better to just throw them out and just keep the whole thing reasonably clean. I'll have a look at it. Perhaps just make it a separate thing (except for the stuff that is used in gromacs/setup.py). Can you open an issue and assign to me, please?

[pslacerda]

Do you have some newbie tutorial for molecular simulation topics?

An approach is to create an executable documentation and verify if the final output is the expected. Some sort of executable markdown would be really nice. It is an tutorial, documentation and also certify code correctness.

If you have some I can convert it in this sort of executable tutorial/documentation/test. Probably to append some bit of of very introductory MDAnalysis (like center of mass or whatever) is a good idea to fully exemplify the GromacsWrapper usefulness.

But first I'm worried about the acceptance criteria of JOSS for binding libraries.

[orbeckst]

But first I'm worried about the acceptance criteria of JOSS for binding libraries.

I don't think that is a worry. The library enabled us to create other packages such as https://github.com/Becksteinlab/MDPOW that has been used in three papers at least. It also contains new code (e.g., in gromacs/setup.py). It is useful for constructing workflows (e.g. in https://github.com/Becksteinlab/mdworks). Only recently has there been made a serious attempt at a Gromacs Python API https://github.com/kassonlab/gmxapi , which looks great but still requires changes in the Gromacs sources. We will certainly cite gmxapi but also point out that the GromacsWrapper approach works with different versions of Gromacs, which allows (version-independent scripts.

[orbeckst]

Do you have some newbie tutorial for molecular simulation topics?

1. Justin Lemkul's (@jalemkul) Gromacs Tutorials, especially the Tutorial 1: Lysozyme in water. Justin is working on a manuscript describing the tutorials in depth, which is available pre-publication as gmx_tutorials.pdf. We could certainly "GromacsWrapper-ify" the Lysozyme Tutorial and then reference Justin's paper and site.
2. Our draft of the AdK tutorial (data at Becksteinlab/AdKGromacsTutorial)) could also be used as a starting point but it has not been fully updated to work correctly with Gromacs 2018 (see the issues).

I recommend (1) to learn how to run a simple simulation with Gromacs 2018.

I should point out that the problems with (2) show potential pitfalls with the claim that GromacsWrapper can write "Gromacs version-independent" scripts because input files and options can still change between Gromacs releases. It would be very interesting to write a mini-tutorial that demonstrates how to take care of these differences with minimal impact. (I think we need a function in GromacsWrapper that identifies the Gromacs release... something such gromacs.gmx_release so that code can make decisions based on the release).

[pslacerda]

Then for sure it pass the criteria!

I'm looking towards (2), a good decision is to warn about using version dependent options, but better to really adapt such options if we could. I also prefer (2) instead of (1) because we can adjust it to our needs.

I'll try to work on the executable (and testable) documentation this weekend.

[orbeckst]

@pslacerda I am tentatively assigning you to this issue. If you can get started with a PR I am happy to chime in. Also feel free to raise issues in https://github.com/Becksteinlab/AdKGromacsTutorial

[pslacerda]

I'm very happy to adapt the tutorial to GW.

The idea is to execute the code blocks in the tutorial, any instructions needed to setup the environment or to test after all the code was executed goes into comments containing such special code.

So we can just run the tutorial and compare the results with the expected. The code excerpts will be moved from the .rst to individual .py/.sh files that will be included in the final .rst files during the build.

Both python and shell version can be choosen like a multilanguage documentation, but for programming instead of natural languages.

Seems a good approach to ensure that the tutorial is running fine, maybe even for other software.

[richardjgowers]

@pslacerda It might be easier to use something like doctesting, which does the .py -> .rst transformation you describe but in reverse. Ie it reads the rst docs and tries to run any code blocks

[pslacerda]

I agree @richardjgowers, would be easier to almost everything. What I'm worried about is to have both tutorials always in sync, would be a bit harder to write a text that cover both languages but is also possible. It also enable to test the original AdKGromacsTutorial.

If you are sure that the tutorials text would diverge too much in order to explain details of GromacsWrapper, than for sure I'd go for doctesting.

[pslacerda]

@richardjgowers @orbeckst Probably use only directives to render only python or shell examples and specific text would be better than what I said.

A test tag (that will never be rendered) can be used to place testing code, and some custom plugin get test and python (or test and shell) code blocks and run, if everything succeeds then the test succeeded.

So we can sphinx-build -t python or sphinx-build -t shell. Will last only how to make a button to switch between the two documentations if really needed.

The evaluation order can be given by the toctree, but probably is a lot easier simply hardcode the rst order there in the testing code.

Example of only directives to render some rst only in the presence of a specific tag: https://stackoverflow.com/questions/42798152/adding-custom-tag-in-sphinx

[whitead]

In my group, for testing and tutorials we use containers (i.e., Docker) so that users do not need to compile or install programs. This also enables use on Windows, Mac, Linux without difficulty. Would that be useful to do the tutorial? That could avoid the complexities of installing gromacs.

[orbeckst]

Docker is one idea. Do you have publicly hosted container images?

For testing of GW we use Gromacs conda packages (4.6.x exists for macOS and Linux, 2018 only for Linux). If we were to package GW as a conda package #109 then we could make the tutorial set-up as simple as (after installing miniconda)

conda config --add channels conda-forge bioconda
conda install -n tutorial gromacs gromacswrapper

(where gromacswrapper and its dependencies would also install numpy, scipy, matplotlib). This would work seamlessly for at least Linux.

I don't think that the conda setup allows one to have multiple versions of Gromacs installed in parallel, so we cannot easily show off having different versions. However, we could write the tutorial so that it works with both 2018 and 4.6 and then the macOS people can also run it directly.

[orbeckst]

@whitead what would be the first steps for setting up with docker, i.e., what would we write in a tutorial?

[whitead]

You could just type:

docker run -it whitelab/gw-tutorial ipython

and it would pull a remote container and bring up an ipython terminal with gromacs and gromacswrapper installed on any OS. We could make an official gromacswrapper docker image instead of using mine (whitelab). To do that, you just write a Dockerfile with the steps to install gw/gromacs. I happen to have one, so the only step would be for you or me to set-up the dockerhub account. You could also maintain specific versions of gromacs in different images. For example, if you want gromacs 4.7:

docker run --rm -it whitelab/gw-tutorial:gromacs-47 ipython

Here's a sophisticated example that uses local files, plotting, and input files with containers.

Here's a close Dockerfile, but like I said I have one specifically for gw that is not hosted currently on github. You can see that by choosing the tag on the gromacs steps (line 24), you could support multiple versions.

[pslacerda]

After some days I got a class that collect code-block directives and concatenate them respecting document order specified in the toctree of index.

The class add callbacks on the doctree-resolved signal to collect code blocks and one instance of toctree, and on the build-finished to wait until all documents be collected and then concatenate blocks to get a full program compiled from all code blocks. The expression doctree.traverse(lambda node: node.tagname == 'toctree') and similar are used to find relevant elements in the documents.

The code-block directives are inside only directives that displays only python or bash code examples depending on sphinx-build -t tagname option.

I.e. all code blocks (only bash or python) get compiled into a program that can be executed after documentation was built. The runnable documentation will continue from there.

I very recently moved to a new house and I'm still without internet on my desktop. So developed on a python2's sphinx installation I had there while studying sphinx on the mobile phone. Don't worry, very soon I'll pull a request into the tutorial repository.

About docker, after properly installed, it is really simple. Even if still have a suboptimal performance in Windows and Mac because its implementation requires virtual machines. What can be more tricky is to make RUN commands that produces what we want. Usually after some apt-get and pip we got the tools we need to do the purpose of the image.

We can work on docker to have a really reproducible and isolated testing environment for the tutorial or even GW.

Sorry if I talked too much for now. I'm really bored in the new house and even more excited to get the tutorial adpatation and then the JOSS paper.