Package documentation

[Omikhleia]

Greetings

SILE 0.10.15 on Ubuntu 20.04

In order to properly document my own package, I tried understanding the autodoc package, and have several concerns.

First, I couldn't find any documentation about it, some made test and trial attempt looking at the SILE manual code. I ended up with the following kind of script (for illustration):

\begin[papersize=6in x 9in, class=book]{document}
\script[src=packages/autodoc]
\script[src=packages/color]
\script[src=packages/rules]

\define[command=code]{\process}
\define[command=note]{\process}
\define[command=dots]{\process}

\bigskip
\package-documentation[src=packages/autodoc] % A standard package

\bigskip
\package-documentation[src=packages/color] % A standard package
\end{document}

Main issues:

• For compilation to succeed, I had to provide (here just as passthru) some commands (= code, note, dots...), whereas one would have expected either the packages to provide the necessary commands, or those to be provided by some supporting package (either autodoc iself or a dependency)... These commands are nowhere to be found in the packages or classes directory.
• The PDF output only shows the documentation for the color package. The autodoc package does have a documentation, though. I suspect it has to do with it including blank lines / white spaces ? If so, shouldn't the code trim the text before parsing it? Several other "standard" packages also fail to have their documentation generated.

All in all, it summarizes to "how to get working documentation for existing packages", and "how should package authors make working documentation"

Secondary issues:

I do feel that enforcing (to some extent) packages to be properly documented improves maintainability and usability. The current autodoc packages looks (no offense intended) as a quick "hack" for the SILE manual. Are there plans or ideas how to correctly handle package documentation? That could include:

• Managing metadata (e.g. author, date, package version)
• Extracting the documented settings and commands (SILE.settings.declare and SILE.registerCommand both have support for help details, so these should be put to use)
• and perhaps other similar things as well.

Regards.

EDIT: Beyond the (small) initial step proposed in PR #1335 ("stage 0"), here is how I would consider subsequent stages... At a slow pace... from simplest ideas to the more complex or challenging:

• [x] An \autodoc:command to typeset commands in a clever and consistent way. First, there are a lot of inconsistencies in the documentation (in the way parameters are presented, etc.). Secondly, it's a bit annoying to have all these escape sequences where we could just rely on the AST parsing to do the job for us... (+ indirect benefits, syntax check)
• [x] An \autodoc:package command for typesetting package names and possibly style them differently. There's no reason they'd always be typeset as "code"...
• [ ] And that \autodoc:package could link to the corresponding location in the manual.
• [x] Commands that should be replaced by roughly equivalent commands defined in the autodoc packages rather than in the Manual (inversion of logic): currently several packages rely on
  • [x] \line,
  • [x] \examplefont
  • [x] and \note.
• [x] Command \code is problematic and has to be reconsidered: It is defined in the url package, but re-defined in the Manual with a less-than-general definition (hard-coded 9pt, etc.). The situation is a bit messy; one may want URLs in regular font but keep the code-formatting option elsewhere, and it's hard to make right currently. (Redefining \url to temporarily kill the \code command is one possible way, but it's not that nice.)
• [ ] Package metadata...
• [ ] More information extraction e.g. use the indexing package to list the concepts mark-up in package documentation, and possibly extract additional bits from settings and commands descriptions (as already suggested above)
• [ ] For in-documentation Lua code snippet, maybe autodoc could implement a naive Lua syntax hightlighting -- it doesn't need additional dependencies (as Penlight includes a (basic) Lua lexer, I used that in my Markdown 3rd-party package)

[Omikhleia]

EDIT: proposed tasks moved to the issue description as per alerque's tip below.

[alerque]

Yes to all points above.

An \autodoc:package command would be especially useful if it did things like link the package name to the corresponding location in the manual.

c.f. #1337 for a related idea that has been kicking around in my head for a while do document internal Lua API stuff rather than just end user facing commands.

[alerque]

Also re information extraction, note our SILE.registerCommand() function has help strings for lots of functions that we basically don't utilize. That might be a jumping off point for more information too. c.f. #1195

[Omikhleia]

An \autodoc:package command would be especially useful if it did things like link the package name to the corresponding location in the manual.

Neat idea indeed. I have added it to my proposed task list above (slightly re-ordered, this is the most probable ordering I'll consider).

[alerque]

Just a GitHub tip: can I suggest you edit the task list above into the original first post in this thread? If you do that GitHub will show the number of complete/incomplete tasks and a progress bar when the issue is references, especially in places such as milestones.