wbamberg
commented
3 years ago I've started this by building a Hugo-based version of the spec. It's currently at https://github.com/wbamberg/matrix-hugo and also deploying to https://adoring-einstein-5ea514.netlify.app/spec/.
Some notes about this.
Content format
I've converted the spec content from RST to Markdown.
Hugo does just about support RST, by calling out to rst2html, but it makes rendering much slower. The big problem though is that ToCs are not automatically generated for RST sources. We could perhaps find a way around this, but converting to Markdown seemed like a better initial approach. I used pandoc to convert, and it was generally pretty smooth, but there is probably a little cleanup to do.
I'd prefer it if we could use Markdown as an authoring format in future.
Page structure
I've kept the overall page structure the same. This means I've kept all the "room versions" as separate pages, kept the specs as single-page, and kept "appendices" as one page. This meant some preprocessing work gluing together all the modules to make the client server spec and gluing together the appendices. I have done this manually for now.
I'd be happier if we:
- maintained all the room versions as a single page
- maintained all the appendices as a single page
- represented each module as a separate page
This would mean we don't have to do any preprocessing of the authored content, and seems like we would get more balanced pages (in particular, the client-server spec would be more manageable). I think there might be more questions around how to structure the spec though, and I didn't want to tackle this just yet.
If we have to, we can surely find a way to preprocess the sources, but the less we have to do the better.
(Another nice thing about representing modules as separate pages is that we could easily use front matter to represent things like which profiles require that module, and we could then have a nice representation of that in the module's page.)
Shortcodes and directives and stuff
I don't yet have any solution for any of the fancy things we do at the moment, like rendering HTTP APIs from OpenAPI specs. These would most likely be implemented as Hugo shortcodes. The theme we are using does include a shortcode for rendering OpenAPI using SwaggerUI, but I don't really think this is a good fit for the spec. SwaggerUI is really for when your website implements a spec and you want people to understand how to use it. But the Matrix spec doesn't implement any APIs, it specifies them.
So there's work in here. I think at least we need:
- rendering HTTP APIs
- rendering events
- rendering spec version information
- rendering the spec change proposals table (although I wonder why we can't just point people at GitHub for this, since it should be possible to present it fairly comprehensibly there).
Random bits and pieces
There are a few things that still need fixing up:
- the home page is terrible
- the footer is not updated
- there are some style improvements we could make to the sidebars
- diagrams don't do too well in the narrower format. It would be nice to replace the ASCII diagrams with SVG.
Also, there's no functional search (but we don't have this on matrix.org, so maybe that's OK?).
ShadowJonathan
richvdh
Current setup
Currently two repos are involved in building and deploying the spec.
The spec is maintained in matrix-doc as a collection of RST and YAML, and a mostly homegrown toolchain builds it into a collection of HTML files. This is then exported to matrix.org, where it becomes input to a Jekyll instance that applies some extra styling.
matrix.org includes a second SSG, Gatsby, that builds and deploys the rest of the site.
Issues
Architectural complexity
It's a complex setup involving two repos and three SSGs (if we include the homegrown one). Although it's simple to generate the docs in matrix-doc, these docs get extra styles applied by the Jekyll instance in matrix.org, and it's not obvious how to see this locally, meaning it's hard to test CSS changes locally.
Homegrown code
Most of the building code in matrix-doc is homegrown, so we have to maintain it ourselves, any changes or enhancements are always dev projects, and anyone starting to contribute has to learn it from scratch. It would be better to lean on existing known projects as much as possible.
Navigation
The spec is hard to navigate. Some pages are huge and the only real nav is a giant table of contents. It's hard then to get some one spec section to another one. It's also hard to get from one page to another.
We need standard nav elements to handle this.
HTTP API rendering
A main piece of the spec is the HTTP APIs that clients and servers use. We have a homegrown template that renders HTML from the Swagger/OpenAPI YAML specification. But the output is not easy to follow. Some of the problems here are minor CSS tweaks, and some are more structural.
Proposal
We'd like to find a new home for the Matrix spec, that we can use as a basis for further improvements.
The option we'd like to explore is:
replace as much as possible of the homegrown code in matrix-doc with an off the shelf static site generator. Were going to try Hugo, because it's fast and flexible and well-established.
publish the spec to a separate site, such as spec.matrix.org
The main risk here we can see will be style/design inconsistencies between spec.matrix.org and the main matrix.org site. So a piece of this work will be ensuring a reasonable consistency.