Automatically create API reference with Sphinx?

[Duchstf]

Since hls4ml's API is rapidly expanding. I thought it would be good for it to be able to automatically compile its API reference too.

For example, I've been learning to use this package and found its API reference very helpful: https://awkward-array.readthedocs.io/en/latest/

Apparently they have a mechanism to just compile the documentation from Python's Docstrings. Some people suggested that it can be set up using Sphinx.

@ngpaladi mentioned that he'll be able to work on this soon, so I'm just opening an issue here for discussion.

[ngpaladi]

I'm currently translating the markdown from the regular webpages to RST to be Sphinx-compatible and I have gotten a Read The Docs theme as in the example working. Getting the autodoc portion fully working and integrating with GitHub Actions will be slightly more complex and may require a bit of time. It also requires everyone to agree on a docstring format (preferably Numpy, Google, or Sphinx formats).

[Duchstf]

Hello,

Do you have the page up already? I think we can just pick a docstring format like NumPy and experiment with it for now. Also, if you have Sphinx doc compilation commands laid out I can write the Github Action for it.

Thanks for working on this,

Duc.

[ngpaladi]

I'm still cleaning up the RST I generated before getting to the autodoc stuff and uploading the page. I don't mind taking care of the GitHub action but it should be as simple as installing the sphinx and sphinx_rtd_theme Python packages then running make html in the sphinx-docs/ folder and pushing the resulting html to the gh-pages branch.

[Duchstf]

Can we make a separate website for the API reference? So we don't have to push the resulting html directly to the gh-pages branch.

[ngpaladi]

We should probably do it through Read the Docs then, but I personally think it's more professional to have all documentation on one consistent website. RST isn't much different than markdown and handles LaTeX a lot better.

[Duchstf]

Yea I'm a bit reluctant to put everything on a page, since I don't have everything thought out yet. I was thinking the main page is for status, some general "how-to"s, command line doc, talks, references, etc. And then the API reference is just something we refer to if we need know the details of a specific function.

So for example, for the the awkward array project, they have three different pages: a main page for general info, a Python API reference, and a C++ API reference. The latter two are compiled automatically. So I thought we can set up something like that.

[ngpaladi]

I can see the advantages of this, but I also feel like if we simplify it by doing the whole site through Sphinx (like many Python packages) we can just put the API reference in a separate "section." It's not that difficult to move stuff over to RST and it cleans up a lot of inconsistencies (like how the site previously dealt with links in a few different ways). Gitbook LaTeX support (at least in terms of the version we use) is also not great, and it would be helpful to have that for the concepts page.

[Duchstf]

Ok great! Let's try doing that then. Could you try to fork the hls4ml repo and we can try to set up the whole deployment workflow from your fork (so we can also see the deployed website from your domain)?

[ngpaladi]

Yep! Already doing that, and I'm planning on getting the site as is transferred to Sphinx first and deploying that before building in the autodoc. I'm testing the procedure with one of my packages (which is a lot simpler): ngpaladi/py2700

[vloncar]

Thanks a lot guys for looking into this.

[ngpaladi]

Ok, so I've recreated the current site with sphinx, and built a copy here. I added the changes from #209 (I was able to get LaTeX support working). The autodoc isn't up and running yet but I am working on that now.

[Duchstf]

Wow, very nice!

[Duchstf]

Can you point me to the source code?

[ngpaladi]

It's essentially the entire sphinx-docs/ folder on the master branch here. The rst files are restructured text files that are essentially the same as markdown with a different syntax. I added the workflow here, which we will just need to modify to point at the correct main repository for the version we merge.

[Duchstf]

@ngpaladi how is markdown syntax different in rst files?

[ngpaladi]

So apparently many people consider markdown to be inadequate for documenting a lot of code, and use restructured text instead. This guide is a pretty useful cheatsheet.

[Duchstf]

ok thanks! Also, so I would imagine the workflow is any commit to master will trigger a build for Sphinx (which will be published in gh-pages branch), is it right?

[ngpaladi]

Yes! That's exactly correct. Just note that the workflow needs to be slightly adjusted when the pull request is made.

[ngpaladi]

This is now a PR: #268

[Duchstf]

Closing issue, any further discussion is moved to the PR.