Add support for reStructuredText

[kaycebasques]

Hi, the Pigweed team is interested in using stardoc. One problem: we use Sphinx and write our docs in reStructuredText. Could you all implement reST support? Or would you be open to a contribution that adds reST support? Sphinx is a very popular tool for generating docs sites, and by extension reST is fairly popular, so I think it could be worthwhile to invest in reST support.

[tpudlik]

@tetromino WDYT?

[tetromino]

Sorry for the very late reply, I was on baby bonding leave.

I'd be happy to review patches for adding reST support (you'd probably want to refer to the current Markdown rendering code in src/main/java/com/google/devtools/build/skydoc/rendering/... for an example) and to answer any questions. Alas, I don't have time to write the reST renderer myself :)

[rickeylev]

FWIW: I ended up doing some pretty heavy sphinx and stardoc integration for rules_python's doc stuff:

• A Sphinx plugin to create a "bzl" domain. This makes writing docs similar as with other programming languages in sphinx.
• A tool to convert stardoc protos to the sphinx bzl domain markup
• Under the hood, it's using MyST. This means you can use Markdown / RST / bzl-domain markup in your docstrings, and it Just Works when it shows up. This lets you do cool things like document the type of a provider attribute and have it xref'd and formatted.

Frankly, I think the result is pretty awesome. It feels so good to be able to use a combination of generated and manual docs that can easily link to each other and external symbols. The manually written API docs are helpful because the bzl domain allows you to document objects that stardoc doesn't natively support (e.g. command line flags). Plus you get Sphinx search, a generated human-friendly index of the code, a generated object index of everything for other projects to consume.

You can check it out at https://github.com/bazelbuild/rules_python/tree/main/sphinxdocs

[kaycebasques]

Cool 😎 Will dive in when I'm back from vacation next week

[fmeum]

@rickeylev Would it be possible to extract this setup into a rules_sphinx? I'm pretty sure that other rulesets would love to adopt it.

[tpudlik]

This is awesome! We build our documentation (https://pigweed.dev) using Sphinx, and we've long talked about doing it using Bazel, but felt a little overwhelmed. Looks like we should take a shot at it!

Regarding extracting into rules_sphinx: since Sphinx is in Python, you probably need rules_python to use it anyway, so breaking this out won't spare anyone any dependencies, will it?

[fmeum]

Regarding extracting into rules_sphinx: since Sphinx is in Python, you probably need rules_python to use it anyway, so breaking this out won't spare anyone any dependencies, will it?

Breaking it out isn't necessary, but it's not clear to me whether this package in rules_python is set up for consumption by other rulesets vs. just a set of internal rules for rules_python's needs.

[rickeylev]

extract this setup into a rules_sphinx?

I'm favorable on the idea. I haven't yet because (1) I don't have time to have to deal with all the surrounding setup (ci, actions, bcr, bla bla; I get all that for free in rules_python), and (2) it's been convenient to edit rules_python docs and doc generator stuff in the same PR, especially as rules_python docs uncover edge cases (I have like 0 tests because testing generated HTML is a PITA, and debugging programs invoked in actions is a bigger PITA).

breaking this out won't spare anyone any dependencies

Yeah, the transitive dependencies would remain about the same. Avoiding the py_binary dependency is tough because Sphinx uses a plugin model.

it's not clear to me whether this package in rules_python is set up for consumption by other rulesets vs. just a set of internal rules for rules_python's needs.

It's both, essentially. "Public but unstable". I've had to make a few changes that would normally be considered breaking. It has public visibility (I plan to use it in rules_test, but there was also a user request to use it, too), and people are free to use it.

using Sphinx via Bazel, but overwhelmed

Yeah. I wouldn't call it hard, but it wasn't easy. Each part is "medium", but there's several pieces that need to be wired together (Sphinx, stardoc, the plugin, surrounding bzl code, subtleties of the RTD environment, probably other stuff i'm forgetting).