CPS pages should be auto-generated

[mwoehlke]

Right now, the 'compiled' pages are generated manually. It would be better if we could automate this.

[bruxisma]

A list of what commands are needed beyond the publish.sh to build the documentation would be useful, as we can then move to a github actions set from there.

[mwoehlke]

@bruxisma, sorry, see also the Makefile (link in edited original description). Which is just a wrapper around sphinx, but importantly it needs to be able to use local extensions. I don't recall if that's tricky, or if publishing credentials are what was tricky, but I'd looked at this previously and something was enough effort that it hasn't already been implemented.

[mwoehlke]

It's worth noting that the current publication method is probably less relevant that how the pages get built.

[bruxisma]

I don't believe that will be an issue as we're force inserting it via sys.insert, and IIRC sphinx has ways to use local extensions without such an approach.

That said, I firmly believe removing the branch approach is necessary for automating the publishing (and previewing, though this is an alpha feature and not currently available to the public) of github pages. I think I could probably knock this out after work-hours later today (assuming Helldivers 2 doesn't grab me by the throat again tonight 😆)

[bruxisma]

@mwoehlke one important question: What's the minimum version of python we're supporting to run the sphinx-build? I want to make sure the tooling I'm adding won't break the world for someone wanting to get involved 🙂

[mwoehlke]

What's the minimum version of python we're supporting to run the sphinx-build?

That's a great question, for which the answer is "I haven't the foggiest". I happen to be using 3.12, though I've almost certainly used whatever was in recent Fedora as of ~2020. conf.py specifies needs_sphinx = '5.3', but that's not the Python version.

I suspect 3.8 or later is fine, and almost certainly 3.10 or later is fine. Honestly, though, my guess is that the Python version is nearly as important as the Sphinx version.

[bruxisma]

I'll set it as 3.10. 3.7 has been deprecated nearly everywhere and 3.8 is next on the chopping block. and I'll make sure sphinx >= 5.3 is in the resulting pyproject.toml file I'm throwing in so we can make sure we have locked down dependencies and a more "hermetic" build in CI 🙂

[bruxisma]

One other question: I can't see much of a difference between the autosectionlabel extension vs the builtin one. Any reason why you'd originally forked it? 👀

[mwoehlke]

I can't see much of a difference between the autosectionlabel extension vs the builtin one. Any reason why you'd originally forked it?

Per the git history, while I'm not certain why it was originally forked, it's likely because it wasn't otherwise available at the time. More recently, it's forked / remains forked because of https://github.com/sphinx-doc/sphinx/issues/11644.

[bruxisma]

Sphinx documentation shows it's been available since version 1.4, but I'll look into the issue you've linked 🙂

[bruxisma]

OK looking at the issue it seems this is slightly related to docutils inability to support nested syntax like markdown supports. Maybe at some point we should consider "not sphinx". But first things first: getting the generation + publishing pushed.

I did note that Fedora is using 5.3, however my upcoming PR is going to suggest using poetry (via pipx) to keep everything in a virtual environment so we SHOULD be able to use a supported sphinx version 🙂

[mwoehlke]

The extension apparently existed in 2017, because I'm fairly sure I didn't write it myself. Note that it's possible it wasn't packaged by Fedora at the time. (It's likely I would have made a local copy rather than relying on pip install.)

The bug is... complicated. On the one hand, someone was unhappy having to match smart quotes in order to be able to link to auto-generated section anchors, which is totally fair. On the other hand, I don't want to have to include the markup when writing such a link. The right solution is for the anchor name to have the markup processed/stripped but not have textual transformations applied. Unfortunately, there is no easy way to do that in Sphinx.

I'm generally a very big fan of Sphinx; reST is more powerful (and better standardized) than Markdown, and Sphinx adds even more power. I'm honestly not aware of any other documentation generation system that comes close to Sphinx in features and abilities. (I've also totally written my own extensions that can handle at least partial nested markdown. :stuck_out_tongue:)

(p.s. I'm currently using python3-sphinx-6.2.1-4.fc39.noarch.)

[bruxisma]

Fair enough. You should see a PR that's open now. I need to get back to other things so ping me if you don't see any movement on it in the next few days 😅

[bruxisma]

if you'd like to test the workflows, you simply need to approve them running 🙂