Manual for the documentation infrastructure

mscheltienne commented 1 year ago

Now that you have an up-and-running infrastructure, let's see how it works.

Infrastructure and versions

Every push to the gh-pages branch is picked up by GitHub which deploys the website corresponding to this branch. https://deep-mi.org/LaPy will bring you to index.html in the root of the branch, here: https://github.com/Deep-MI/LaPy/blob/gh-pages/index.html This landing page is a simple redirect to https://Deep-MI.github.io/LaPy/dev/index.html which corresponds to https://github.com/Deep-MI/LaPy/blob/gh-pages/dev/index.html But why a dev, stable, and other folders? To keep the documentation of different versions.

dev should contain the documentation from the current main branch. It is updated every time a push on main is done by the documentation workflow
stable should contain the documentation from the latest released version. At the moment empty, and it should be updated when the next release occurs by renaming dev to stable and creating a new empty dev directory. Note that this part is not automated in the current workflow, but it could be by editing the publication workflow.
0.x or whatever the version is for older version of the package.

Concrete example with MNE-Python: https://github.com/mne-tools/mne-tools.github.io You can find the documentation from version 0.11 to the current 1.5 dev version.

With this folder system, you can access:

dev at https://Deep-MI.github.io/LaPy/dev/index.html
stable at https://Deep-MI.github.io/LaPy/stable/index.html
0.10 at https://Deep-MI.github.io/LaPy/0.10/index.html

and so on. It's also very common to set up a version selector on the website to change the version easily, or a banner to redirect to the stable doc when you are looking at the dev doc. e.g. for nilearn: https://nilearn.github.io/dev/index.html How do you add a banner only for the dev documentation? The website has no knowledge of which version you are looking at. It's static. But, when you build the documentation for a given version with sphinx, you can adapt the build based on the version. e.g. by adding a banner if the version of the package documented has dev in it: https://github.com/nilearn/nilearn/blob/6085633bc3ffed18feea47ff2e39c9f5fbebcfd5/doc/conf.py#L199-L208

Document your code

sphinx will not document all your functions, classes. Only the one you tell him to document. This is done here, in a section commonly titled API: https://github.com/Deep-MI/LaPy/tree/main/doc/api The directives used to document your code are:

.. currentmodule:: lapy

.. autosummary::
    :toctree: generated/

    TriaMesh
    TetMesh

.. currentmodule:: to tell sphinx where to look for. Here, it's equivalent to from lapy .. autosummary:: to tell sphinx what to look for.

Thus in the example above, conceptually, it's equivalent to from lapy import TriaMesh, TetMesh. One of the best way to improve the documentation is to improve the docstrings. You can use different sections, e.g.

Parameters
----------

Returns
-------

Notes
-----

References
----------

Example in MNE-Python, ICA class: https://mne.tools/stable/generated/mne.preprocessing.ICA.html#mne.preprocessing.ICA and the corresponding docstring: https://github.com/mne-tools/mne-python/blob/e5b217ecaf256d13d18d2784010824673660d25e/mne/preprocessing/ica.py#L206-L438

Document using rst

sphinx will also build documentation from additional files in the doc folder. For instance, the changelog: https://github.com/Deep-MI/LaPy/blob/main/doc/changes/index.rst This changelog points to the file latest.rst: https://github.com/Deep-MI/LaPy/blob/main/doc/changes/latest.rst Which contains the changes of the current dev version. Usually, you would include one file for each version, for instance in MNE-Python: https://github.com/mne-tools/mne-python/tree/main/doc/changes (note the different format, not that it matters :wink:).

You can include any number of additional documents. For instance in MNE-Python, a separate page detailing the Installation: https://mne.tools/stable/install/index.html The URL points you to the corresponding files in the doc folder, doc/install/index.rst: https://github.com/mne-tools/mne-python/blob/main/doc/install/index.rst

Add items to the main sidebar

The sidebar on the left of the website https://deep-mi.org/LaPy/dev/index.html is defined by the toctree on the landing page index.html: https://github.com/Deep-MI/LaPy/blob/main/doc/index.rst If you add additional documents, e.g. installation instructions, don't forget to include this document in a toctree. If it is not included in any toctree, sphinx will issue a warning during the build. This warning can be silenced by setting this document to an "orphan" with the directive :orphan:.

Add tutorials/examples

A big asset of sphinx: you can add extensions. For instance, sphinx-gallery which lets you build tutorials and examples. In the configuration, the directory listed is "../tutorials" which corresponds to https://github.com/Deep-MI/LaPy/tree/main/tutorials Each .py file in this folder starting with the filename_patern r"\d{2}_" will be interpreted as an example. For instance, 00_my_tutorial.py. The 2 decimal naming convention let's you start by creating the tutorials 00, 10, 20, ordered numerically, and add later a tutorial between 00 and 10, e.g. 05.

Example with MNE-Python:

Tutorials: https://github.com/mne-tools/mne-python/tree/main/tutorials The tutorials are grouped by categories in subfolders, e.g. preprocessing: https://github.com/mne-tools/mne-python/tree/main/tutorials/preprocessing

The tutorial format is a mix of jupyter-notebook and .py:

# %%    <- defines a new section
# text
# more text
# as long as the following lines start with a #

some code
# this is now a code comment because the # is not following a new section

# %% <- a new section
# more text

For example with this tutorial: https://mne.tools/stable/auto_tutorials/preprocessing/55_setting_eeg_reference.html#sphx-glr-auto-tutorials-preprocessing-55-setting-eeg-reference-py https://github.com/mne-tools/mne-python/blob/main/tutorials/preprocessing/55_setting_eeg_reference.py

Use extensions!

The sphinx extensions in-use are defined here: https://github.com/Deep-MI/LaPy/blob/1079a4dd4fefcc40949cbbde6d3d05761a46c17b/doc/conf.py#L39-L51

You can use a bibliography .bib to manage your references, you can use sphinx.ext.mathjax to render equation, ... sphinx and all the extensions are configured in doc/conf.py.

And finally, add a link to the website in pyproject.toml, in the readme, ... ;)

It's a big dump of information, but I hope it will be useful to you. The best way to learn more about the possibilities offered is to play with the files, try to change something and build the documentation. You can either let the CI build the documentation in a PR and retrieve the artifacts; or you can build the documentation locally (much faster). Simply navigate to the doc folder and run make html.

m-reuter commented 1 year ago

Thanks for the great overview. I already have one question: is it possible to have also https://deep-mi.org/lapy to be forwarded (not only the .../LaPy ) ? I don't like capital letters in urls (and case sensitivity).

mscheltienne commented 1 year ago

It's definitely possible and it would be nicer, but I don't have precise instructions on how you can change the domain of a gh-pages website. In the settings, it's likely in Pages -> Custom domain. There is this documentation which might help: https://docs.github.com/en/pages/configuring-a-custom-domain-for-your-github-pages-site

m-reuter commented 1 year ago

@mscheltienne the doc build seems to be broken:

Extension error (furo):
Handler <function _html_page_context at 0x7f84051221f0> for event 'html-page-context' threw an exception (exception: This documentation is not using `furo.css` as the stylesheet. If you have set `html_style` in your conf.py file, remove it.)
writing output... [  4%] api/generated/lapy.Solver
Error: Process completed with exit code 2.

I did some changes to conf.py (mainly reformatting and comments), but even reverting them to the last working state did not help. Do you know how to fix this? It is the same in our other repositories (BrainPrint and WhipperSnapPy) although there we are trying to build doc for the first time.

m-reuter commented 1 year ago

Thanks for #40 but after merging we still get the furo error.

mscheltienne commented 1 year ago

Now it's working :) I had to do a similar pin on sphinx for my builds, but I could work with the latest furo version. Sphinx-gallery also released an update today fixing bugs with sphinx 7.2; they definitely broke a couple of packages/extensions with that release.

Deep-MI / LaPy