Markdown trial

[sthagen]

Why?

From the standpoint of a technical editor our private working draft editor copy workflow from which we periodically publish revisions as snapshots is far from ideal.

Problems:

• the evolution is hidden from the public between publication of snapshots
• initially and again section numbering errors in the google hosted document
• wrong formats hidden in the proprietary document format used on hosting service
• extra effort for editors to derive the snapshots and place those on OASIS hosted systems for public scrutiny and archival
• hard to follow commenting and provisioning of alternatives in the text
• tables often used as blind tables
• bitfield display through tables with optional bytes is not optimal

In case we can transform the google hosted document into a maintainable and provably correct source format like markdown inside out publicly accessible MQTT TC repository (here) all these aforementioned problems go away (and a few new ones will for sure join us 😁).

Why now?

Well, the TC members are in deep but very content local discussions on the details of the protocol, which means the overall structure of the document can be considered stable.

What and How?

• Slicing the large document into many section-aligned separate files (level of cuts selected per level 1 section to balance the content storage per file).
• Separating the concerns of source and delivery items
  • many markdown source files with a bind file determining the order
  • deriving section numbers (TBD) from the position in the content tree
  • enabling search for optimal bitfield illustrations
• Delivery items:
  • large single markdown file to be rendered online at GitHub (TBD)
  • single html file with all figures included as data (TBD)
  • PDF rendition (TBD)
• The three delivery items should be 1 to 1 publishable by OASIS administration

Note:

This change set is not ready yet for review and merge but instead it is a sneak preview.

How Far is the Transformation Progressed?

Shopping list:

• [x] frontmatter
• [ ] table of contents (auto-numbering)
• [x] introduction
• [x] control packet format
• [x] control packets
• [x] operational behavior
• [x] conformance
• [x] references
• [x] security and privacy considerations
• [x] acknowledgements
• [x] revision history
• [x] backwards compatibility
• [x] implementation notes
• [x] notices
• [ ] update to latest content from opaque vendor platform
• [ ] cross-sectional tasks
  • [x] verify mermaid diagram embedded source code in markdown renders on GitHub
  • [ ] auto-numbering of sections
  • [x] ensure unique section labels across the assembly
  • [ ] auto-numbering of figures
  • [ ] auto-numbering of tables
  • [x] optimal bitfield description format
  • [ ] implementation of semantic referencing
  • [ ] rendering of delivery items
  • [x] single-file markdown+layout_and_navigation_hacks
  • [ ] self-contained html file
  • [ ] pdf file

The delivery item single-file markdown will be enhanced with navigational fluff and coated with display sugar, but already the rendered view of the source files (those with content) on the source branch should give some impression how close we will be when the transform based on the upstream snapshot from 2024-MAR-01 will be completed.

Examples:

• Section 1.1 MQTT For Sensor Networks (MQTT-SN) - introduction-01-the-specification.md
• Section 1.5 Data representation - introduction-05-data-representation.md
• Section 2.2 Packet Identifier - control-packet-format-02-identifier.md
• Section 2.3 MQTT-SN Packet Fields - control-packet-format-03-packet-fields.md
• Section 2.4 Topic Alias Types - control-packet-format-04-topic-alias-types.md
• Section 4.4 Session Establishment - operational-behavior-04-session-establishment.md showcase how GitHub directly renders mermaid source code blocks nicely Section 4.18 - Sleeping Clients operational-behavior-18-sleeping-clients.md

The transformation is performed by trying to remain as automatic as possible, so that we can repeat the process for a few future snapshots without too much overhead.

Also, there are transformation notes left in the markdown sources where something is not looking correct, or where -in the opinion of the transformer- there should be a better or an already planned future way that would be considered a complete transform.

Example for better ways: blind table use, table use to mimic bitfields.

Example for future planned additional changes: Switch to auto-numbering.

To see these transform notes (documented as html comments, one can simply switch to the "code" or "raw" view for above example URLs.

As an example:

"Code" button selected or manually the query string ?plain=1 appended to the URL: control-packet-format-04-topic-alias-types.md?plain=1

[sthagen]

The insert of the packet diagrams as figures awaits the merge of the PR #21 ...

Update: packet diagrams are now available (PR #21 has been merged)