The documentation situation

[Anaminus]

This issue tracks the progress of the current chaotic state of documentation for rbxmk.

Program

Documentation from within the program is currently expressed through "fragments", which are snippets of content embedded into the executable. They currently live here:

https://github.com/Anaminus/rbxmk/tree/imperative/fragments

Work here is still very much in progress. Further iteration is on-hold while the documentation website is developed.

Website

A work-in-progress website for documentation currently lives here:

https://anaminus.github.io/RBXMK_DOC_TEMPORARY_DO_NOT_USE_OR_YOU_WILL_BE_FIRED/

Unified theory

Currently, the program and website source their content separately, even though they contain mostly the same information. It would be much better if a single, unified source were used instead. For now, they will continue to be worked on separately until a common system emerges.

Working draft

• The input is some sort of query. There are several types of query:
  • doc: A path matching an organized structure that enables querying of general topics in addition to libraries, types, etc. Likely similar to a documentation website.
  • api: A dot-syntax path matching the Lua environment.
  • library: A dot-syntax path matching a library. Goes by individual library name instead of merged environment.
  • format: Matches the name of a format.
  • type: Matches the name of a type.
• A query is resolved into a set of fragment references.
• Fragments are embedded in the program.
• Fragment structure involves directories, files, and sections.
• Fragments are agnostically structured; a single file with a number of sections is effectively the same as a number of files each with one section. The only difference is how the fragment is referred to.
• A section is delimited by a section directive.
  • Currently, this is <section name="...">...</section>
  • May try completely differently syntax to avoid ambiguities and reduce need for escaping.
• Program can query fragments by reference, which includes drilling into them.
  • This involves parsing the fragment file, if only to get a slice of it.
  • A fragment slice is treated the same as a fragment file. Program does its best to make sure there are no formatting problems caused by reencoding.
  • The section parser only sees section directives; other content is left alone.
• Sections can specify a format of their content.
  • Inner sections use same section directive regardless of format.
  • Determines how the content is parsed, to be converted into raw format.
  • Specified as <section format="...">. Format name is a file extension.
  • Extension of file determines the format for the entire file.
  • More generally, the <stem>.<ext> of a fragment file name maps to <section name="stem" format="ext">.
• Fragment content is parsed based on format.
  • Converted to "template" format. This is effectively a Go HTML template.
  • Section directives, regardless of what syntax we decide on, will be converted to <section> tags. Only the name attribute is retained.
  • Content is always wrapped in a single section element. For a full document, this element takes on the name of the file.
  • Care is taken to ensure template directives are retained.
• Template format contains template directives.
  • Template is executed to process directives.
  • Directives insert content in Raw format.
  • Example: frag template function.
    • Function outputs content in Raw format that is inserted at directive location.
    • Evaluates recursively.
• Having executed the template, the content is now in the Raw format.
  • This is the final intermediate format, ready to be processed by a formatter.
  • Can contain non-standard elements that represent hints to formatters.
  • For example, a <toc> element hints at an appropriate location to place a table of contents.
  • Formats that don't need a hint can just ignore it. e.g. The html format will skip rendering a <toc> if it's only rendering a subsection. Or, if it's rendering a whole document, replaces the hint with an actual ToC.
• Raw content is processed by formatter.
  • HTML: Presented as a complete HTML document.
  • Terminal: Presented in a format suitable for display in the terminal. Utilized by the program.
  • Text: Extremely stripped down format. Suitable only for sections with basic content. Used by the program for placing text for programmatic purposes, rather than emitting directly to the terminal. e.g. error messages.
  • DocSite: Raw content is output directly to an external "translator", which processes the content to be incorporated into a documentation website. e.g. Hugo-generated static site with a theme.
  • User: Content is presented directly to the user as-is.

Several inputs are provided by the user. The combination of these inputs determine the strategy used to generate the output. Some examples:

Query	Output	Format	Result
CFrame type	File	HTML	An HTML file with a self-contained description of CFrame the type.
Empty	Directory	HTML	A complete multi-paged cross-linked HTML website generated under the specified directory.
Empty	File	HTML	An entire website with all information contained within one file.
math library	Unspecified	Terminal	A description of the math library, printed to stdout in a format suitable for the terminal.
Empty	Directory	Raw	The raw content of all fragments are dumped to the specified directory.

In the last example, the raw content is dumped. This can be passed to a "translator" that converts the content into a format suitable for a documentation site. Because there are an endless number of ways to produce a website, the translator lives next to the website instead of near or within rbxmk.

The current documentation website is a static site generated by Hugo with the Shadocs theme. The translator is a program that receives the raw content from rbxmk and formats it for this configuration of static-site generator and theme.

For fun, here's a graph:

[Anaminus]

After resolving a query into a fragment reference, an option specifies whether to return that reference directly. This is useful for converting a query to a canonical form.