[WIP] Add a sphinx API documentation

[jbarnoud]

API sphinx documentation

This pull request introduce the framework to document the API using sphinx. It aims at having an extensive documentation available on read the docs.

The doc is, for now, splited in two parts: one part are jupyter notebooks that describe how to use the API to perform certain tasks, these notebooks links to the second part that is a reference for the API build upon the docstrings.

Notebooks

The notebooks are stored in the doc-api/source/notebooks directory. During the build, the notebooks are executed, then they are converted to restructuredtext files that are used by sphinx. The build script ad a header to each notebook with the table of contents, and with links to alternative formats for the notebook. The links do not work on Read the docs yet, but they work on a local build.

Links to the reference are made by using the rst syntax within the notebook. See the documentation for inline cross reference in sphinx. Only the inline references can work.

Note that more rst construct could be used using raw notebook cells. Yet, an issue in recent versions of pandoc prevents the direct conversion of the notebooks to rst. The work around is to use a conversion to markdown as an intermediate, this prevent the use of non inline rst constructs.

The build will fail if a notebook finish with an empty cell!

API reference

The API reference is build based on the module docstrings. Docstring are written in the numpy format for docstrings that we used so far. They are supported by sphinx thanks to the napoleon module.

Note that the keyword to list the exceptions raised by a function is Raises rather than Exceptions.

File organization

The source files for the doc are placed in doc-api/source. In doc-api/source/pages are pages uses for the reference. Notebooks are in doc-api/source/notebooks. The root page is build based on doc-api/source/index.rst.

The doc-api/source directory also contains files needed to build the doc. conf.py is the configuration for sphinx; it calls build.sh that handle non basic transformations prior to sphinx regular tasks.

The Fontmap.GS file is a work around a bug in Read the Docs. See the commit message of 7c920e8f55e5da6c1617d69487e3b6eaa21e9b57 for more details.

Building the doc

The doc should be built using python 2 as it requires MDAnalysis that is not yet fully compatible with python 3. All requirements of PBxplore must be installed so as extra ones that are listed in doc_requirements.txt. All these requirements can be installed by running pip -r doc_requirements.txt. Note that MDAnamysis must requires numpy even to setup its install. Therefore, MDAnalysis cannot be installed in the same step as numpy with pip. For this reason, MDAnalysis is installed uning pip during the building of the doc.

To build the doc, run make html in the doc-api directory.

Read the docs

The sphinx doc aims at being build by Readthedocs. The PDF generation does not work, and thus should be deactivated in readthedocs advanced settings. Also, the system site-package should be make available to the virtualenv to access matplotlib. Finally, the requirements should be installed from doc_requirements.txt.

Perspectives

The doc can now be improved iteratively. Some things are to be fixed:

• the links to other version of the notebooks (direct html render and raw notebooks) are broken
• some typo break the cross references

---

Original message:

See #81

This pull request setup sphinx to document the API. Sphinx is setup to render a reference documentation using the docstrings. The docstring are formated using the numpy format. So far we were making an error regarding this format: the keyword to list the exceptions raised by a function is Raises rather than Exceptions.

The pull request adds docstrings in several __init__.py files. This could conflict with the cleaning @HubLot is doing with optional dependencies.

In order to build the doc, run make html in the doc-api directory; then open doc-api/build/html/index.html. Sphinx and napoleon are required to build the doc; see the dev_requirements.txt file.

[HubLot]

:+1:

[jbarnoud]

Here is a demo, because building the doc is not trivial. http://jbarnoud.github.io/PBxplore/index.html cc @pierrepo @HubLot @alexdb27

[pierrepo]

@jbarnoud and @HubLot thanks for your efforts. If I understood correctly, we have two choices for the documentation:

1. We use Sphinx and we have to build ourselve the documentation based on doctrings. The doc is then available on GitHub
2. We use readthedoc that automatically build the documentation after every commit as a CI. The doc is then available in the readthedoc website.

Option 2 seems easier to me. No? I can quickly create an account at readthedoc if needed. @jbarnoud you can hate me ;-) Let me know what you think about that.

[jbarnoud]

@pierrepo That's pretty much it. There is a third option that is pythonhosted.org which comes with pipy. But I do not know how it works.

We use Sphinx and we have to build ourselve the documentation based on doctrings. The doc is then available on GitHub

It should be possible to have the doc built by Travis and pushed to the main repo gh-pages branch. This way, it is fully automated. Yet, it is not the most trivial way. It could be the best option, though, as building the doc requires several external tools due to the inclusion of notebooks.

We use readthedoc that automatically build the documentation after every commit as a CI. The doc is then available in the readthedoc website.

This would be the simpler option, assuming we can have readthedocs build the doc. As I said above, the build requires a bunch of tools (jupyter and pandoc mostly), and these tools may not be available in the context of readthedoc.

Option 2 seems easier to me. No? I can quickly create an account at readthedoc if needed.

I will try to have readthedoc build our doc on my fork. If I manage to do it, then I'll explain how I did it.

@jbarnoud you can hate me ;-)

Do you really want me to hate you? :stuck_out_tongue_winking_eye:

[jbarnoud]

@pierrepo Readthedocs is a viable option. I managed to have it build the doc dispite the apparently complicated building process : http://jbarnoudpbxplore.readthedocs.org. I have a lot of cleaning to do, though.

By the way, I added a notebook about writing PB sequences in files. It lead to #85. http://jbarnoudpbxplore.readthedocs.org/en/latest/notebooks/WritePB.html

[pierrepo]

OK nice! I will open an account at readthedoc. Which tutorial did you follow to setup your account? For me, we need to kind of documentation:

• code documentation extracted from source files
• tutorial / how-to documentation extracted from ipython notebooks

Sounds good to you too?

[jbarnoud]

I followed no tutorial. I just created an account, connected it to my github one, activated the repo in the readthedocs interface, and touched all the buttons until it worked :P

More seriously, the 3 first steps are required and should be trivial. Then some settings in the advanced settings are necessary:

• Requirements file must be set to doc_requirements.txt
• Python configuration file can be blank
• Default branch must be the branch you want the doc to be build against, I would strongly advice to try with an other branch that master at first
• The PDF version does not work and must be deactivated, I do not know for the epub version
• The option " Give the virtual environment access to the global site-packages dir." must be selected

Once the settings changed, do not forget to click "submit" (it became so frequent that settings get saved as you change them that I did not even notice the submit button).

Note that the build process is heavily customized and therefore most likely do not look like the tutorials. For instance, the build workaround a bug in nbconvert/pandoc, and it adds a header in the notebooks with the table of content and links to other versions of the notebook.

On 08/10/15 12:09, Pierre Poulain wrote:

OK nice! I will open an account at readthedoc. Which tutorial did you follow to setup your account? For me, we need to kind of documentation:

• code documentation extracted from source files
• tutorial / how-to documentation extracted from ipython notebooks

Sounds good to you too?

— Reply to this email directly or view it on GitHub https://github.com/pierrepo/PBxplore/pull/82#issuecomment-146484112.

[pierrepo]

Thanks. Why not directly plug readthedoc to the master branch? I least we will see the mess from the beginning and will improve it PR after PR.

Do we still need the Sphinx doc?

[jbarnoud]

The doc on readthedocs is the sphinx doc. So yes we still need it :)

I think we should start on a different branch because master should never be broken, yet the doc still requires some cleanups that could lead to a break. The history of my sphinx branch is super messy so I could even do a rebase to clean it, and master should never go through a rebase.

[pierrepo]

Thanks. Why not directly plug readthedoc to the master branch? I least we will see the mess from the beginning and will improve it PR after PR.

Do we still need the Sphinx doc?

[jbarnoud]

I am fighting with ReadTheDocs that fails with weblogo because of font issues in ghostscript. Once this issue solved, I will clean my history, and the branch will be ready to merge. Then @pierrepo can set ReadTheDocs on master as there should not be drastic changes anymore.

[jbarnoud]

Argh. @pierrepo you are quick to merge! Not long ago the pull request was full of crocodiles, hence the work-in-progress label. Thankfully this one was just fixed.

Tomorrow I'll update the pull request message to document few things to know about how the doc works.

[jbarnoud]

I updated the pull request message to document the pull request, even so it is merged already. I also linked issues on things I need to finish.

[pierrepo]

Oups. Sorry @jbarnoud. I misread your messages and thought I had to quickly merge your PR :-( Let me know when you're ready.