Create a design specification for RriegsWiki

[rriegs]

All software needs a design specification! In this case, we need to flesh-out the idea of a single-page, small-content wiki and then take a look at the required components of said. To get things started:

• Single-page means that moving between topics should not incur page loads, but should instead dynamically update page content based on RESTful query responses or internally maintained state.
• Small-content means that topics should be encouraged to be short and reusable. It is thus necessary for topics to be easy to create, split, link, and view.

TiddlyWiki fulfills both of the above goals particularly well, and indeed RriegsWiki draws inspiration primarily from it. Nonetheless, I am motivated to implement my own wiki due to various qualms I have with TiddlyWiki's design. I'll not detail those now, however, because, rather than dwelling on what (I think) is wrong with TiddlyWiki, this issue is meant to collate and analyze what might be right for RriegsWiki.

[rriegs]

Back end

First things first: RriegsWiki is a browser-based application, and as such necessarily has a front end. Considering the single-page design goal, this front end will be driven by JavaScript. An open question, however, is what to do about the back end.

TiddlyWiki is split on this decision and in fact offers two editing experiences because of it: one, a client-side-only editor that can only save changes via manual export of all wiki content and, two, a client-server setup that automatically saves changes as they are made. As I see it, the latter is far preferable to the former, given the nuisance of manual checkpointing and the propensity to lose data as a result. The former's only real wins are the ability to edit (and copy) otherwise static wikis and the absence of any kind of back-end configuration (indeed, the entire wiki software and content are stored in a single file).

I have other ideas concerning the forking and modification of static wikis (to be discussed below), and while I find the self-contained single file neat, I value reliability and functionality above the absence of a back end. For browser-based front ends, it seems that the easiest (and best supported) kind of back end is one providing a REST API. Beneath that, one can use anything, but the Node.js environment appears to offer the broadest and most easily configured toolkit. (It's what TiddlyWiki uses, after all.)

[rriegs]

Back end functionality

The most basic form of RriegsWiki's back end needs only support a very few verbs:

• GET: serve topic content in the body of the response (just like any webpage)
• PUT: create or replace stored topic content
• DELETE: remove a stored topic

One can imagine uses for POST such as renaming or cloning topics, but even that is optional.

A natural choice for topic storage is the file system itself, though this presents some potential downsides. For one, the minimum file size is usually around 4k, which can bloat the footprint of an otherwise small-content wiki, though this is unlikely to matter much. Of greater concern is the potential to accidentally (or maliciously) perform fs operations that might somehow compromise the host.

A safer alternative may be to keep wiki topics in a database. Even SQLite should suffice, and would have the added benefit of being transferable via simple file copy. That being said, avoiding the file system precludes the ability to use Git as an edit history management system.

For now, the file system seems to be the easier and more functional choice for topic storage, so that is the direction I will pursue. This again mimics a design decision of TiddlyWiki.

[rriegs]

Back end framework

Even having settled on a Node-powered back end, there are many candidate frameworks and other utilities to choose from in developing the server. Node's built-ins are powerful enough to write your own web server in just a few dozen lines, but their asynchronous forms must be invoked via dizzying chains of callback functions and I suspect many pitfalls await the incautious.

The most popular third-party framework by far seems to be Express, though numerous others exist with varying degrees of overlap. Most tend to be heavyweights and to depend on many additional packages, but that's not to say they're slow: they're geared towards massive, complex installations with thousands of simultaneous users. RriegsWiki, on the other hand, needs only serve one or a few users at a time. Accordingly, I prefer simplicity over (unneeded) functionality, and for that, Micro seems to fit the bill.

In addition to being small and nimble, Micro is specifically designed to make use of Node's recently added support for the async/await pattern. For some time, Node has included Promises as an alternative to callbacks, and more recent versions have added the async/await syntactic sugar for these. Unfortunately, many Node modules like fs have yet to add native Promise support. That being said, the fs-extra package offers an updated API addressing this omission (amongst a number of others). It's one of the most popular packages on npm and appears to be actively maintained, making it preferable to the piecewise alternatives.

[rriegs]

Front end

There are plenty of things to think about when developing the front end of any web application, and first among these is handling users with JavaScript disabled. TiddlyWiki is fairly inept in this case, but MediaWiki is almost unaffected (as would be expected for any site relying primarily on server-side PHP rendering). It's fun to imagine how RriegsWiki could function even without JavaScript, e.g. by instead implementing much of its functionality in pure CSS, but that will have to remain a project for another day: avoiding JavaScript would forfeit the goal of being SPA. (Regardless, it would be nice to at least show some content in the absence of JavaScript.)

Another chief concern in front end development is accessibility. It's tempting to ignore this at first, though doing so may seriously complicate later efforts to address it in the future. Regardless, I plan on using conventional HTML to render all of the user interface (i.e. no canvas element, etc.), with JavaScript only handling client-server communication and the generation of said HTML. As such, screen readers and keyboard controls (Tab, etc.) should remain viable with a minimum of additional effort (OK, maybe not so minimum).

[rriegs]

Front end functionality

The front end's job is to render wiki content, navigate between topics, and facilitate the editing of said topics. While I've already been using the term, I should formalize:

• A topic is the fundamental unit of wiki content, i.e. what to TiddlyWiki is a tiddler and to MediaWiki is a page. Topics have unique titles and may store a variety of data types in their bodies.

Again like TiddlyWiki, RriegsWiki should be able to display more than one topic at a time. Specifically:

• Multiple open topics should be visible in a vertical, "long scroll" arrangement
• Topic content should be written in a familiar, intuitive, and expressive markup language
• Clicking on a link to a topic should add it to the list of open topics or scroll to it if it is already open
• When editing/creating a topic, both the source and current rendered result should be visible at once
• The editing experience should be devoid of needless clutter, focusing instead on content

This is, of course, just the beginning. Some secondary features include:

• A search box to open topics by exact title match or keyword search
• The ability to "shade" and rearrange open topics
• Non-hierarchical categorization via tags

Yet more advanced functionality will be discussed below.

[rriegs]

Front end framework

As noted, the RriegsWiki front end will be made up of the typical combination of HTML, CSS, and JavaScript, and I have no problem with insisting on JavaScript being enabled for full functionality. I do, however, have a bit of a problem with JavaScript itself: the language is riddled with pitfalls, to the point that debugging even simple apps is near hopeless without years of experience and an extensive testing framework.

To my relief, numerous alternatives exist to mitigate JavaScript's various problems. Of these, I'm most intrigued by Elm, which, in a nutshell, is Haskell for the web. Among Elm's numerous desirable properties is its guarantee never to throw run-time exceptions; such bugs are caught at compile time due to its strong type checking and enforced error handling.

Elm is also apparently pretty fast, and employs an easy-to-develop application architecture. Another nice property is Elm's interoperability with existing JavaScript, which will prove useful for another one of RriegsWiki's goals: sane markup (see next comment).

[rriegs]

Wiki content makup

The current leading, legible markup language is Markdown, which I am strongly inclined to use as opposed to creating my own. (MediaWiki is an acceptable alternative, though clunkier in comparison. TiddlyWiki, however, uses a curious amalgam of slightly-off MediaWiki and other syntax, which is one of my least favorite aspects of the platform.) Alas, Markdown has many different flavors and yet more different parsers for its various contexts and forms.

I will require client-side rendering and thus a JavaScript-based parser. As for flavor, CommonMark looks to be the best specified; accordingly, markdown-it seems to be the natural choice. It may not be as fast as some other Markdown parsers, but markdown-it also features a versatile configuration system with many official (and unofficial) plugins already available.

The plugins I'm interested in chiefly add syntax for HTML tags not considered by CommonMark. I may also want to craft my own plugin for simplified wiki links: CommonMark has no syntax that derives link destinations from the link text itself, which is common practice in wikis when referencing other topics by title. MediaWiki-style links happen not to conflict with any CommonMark syntax, and so seem to be a suitable model. A plugin already exists for this, but it's handling of MediaWiki link functionality is incomplete.

[rriegs]

Advanced features

Just the above is enough to count as a feature-complete wiki, but not necessarily one with all the creature comforts one might expect coming from a mature environment like MediaWiki. Some common additional features include:

• Topic edit history detailing who, when, what, and why
• Special pages such as recent changes and "what links here?"
• Appearance and other settings, including per-topic settings
• Templates and transclusion
• User management

TiddlyWiki notably does not maintain tiddler edit history, nor is it designed for multi-user editing. MediaWiki has all of the above except for per-page settings.

[rriegs]

Revision history via Git

Git is well equipped to assist with several of the above features: revision history, of course, but also user management and providing content for some of special pages of interest. As suggested above, it would also be useful for forking otherwise static wikis, with the added benefit of enabling later pull requests to incorporate specific edits made. There is already at least one wiki platform, Gollum, built around Git as the primary content management system.

Gollum doesn't expose some Git functionality that is nonetheless commonly used in software development, such as commits that coherently modify multiple files at once and the ability to branch and merge. I suspect Gollum's developers have good reason not to include these features, however: one-commit-per-edit permits them to push edits live immediately and, perhaps more importantly, reduces the complexity of the editing experience. Forgetting to add changes to a commit and forgetting to push commits to remote are two all-too-easy blunders for Git users both new and experienced. Always committing and pushing immediately upon saving changes eliminates the possibility of this kind of mistake.

On the other hand, I wonder if this practice is optimal for a small-content wiki. In RriegsWiki, it should be common to edit multiple topics at the same time, or indeed to create multiple new topics when editing another. Perpetually committing changes would seem to bog down history, but then again, commits are cheap, and if small-content favors lots of tiny topics, why not lots of tiny commits, too? I might instead reserve the changed-but-not-committed state for autosaved files in the process of being edited; this would permit the resumption of an interrupted session without much confusion.

I'd still like to introduce a notion of branching, such that large reorganizations, etc., can be made atomically, permitting easy reversion (if desired) or the ability to halt work on the task while resuming edits elsewhere. That one is working in a branch should be visually obvious (e.g. by changing the background color), and branching should not be encouraged for routine wiki editing.

[rriegs]

Git-related verbs

Incorporating Git-based revision history will require several changes and additions to the verbs exposed by the back end:

• GET: (maybe) extract a topic's most recent edit date, author, and hash
• PUT: also commit changes with a provided or derived message (unless autosaving)
• DELETE: git rm and commit
• rename: git mv and commit
• history: return git log for a given topic
• compare: return git diff for a given topic and revisions
• revert/reset: revert some specific change or reset a topic to some previous version

Verbs extending beyond the functionality of Gollum are as follows:

• branches: list available branches
• checkout: switch to or create a branch; alternately, checkout a specific hash
• merge: git merge, somehow handling conflicts
• delete branch: confirm if not merged, then git branch -D

Properly handling conflicts will likely require another verb or two.

The front end will need to exhibit more behaviors as well:

• Show a topic's commit history with buttons to compare, revert, and reset
• Show a comparison between two revisions for a given topic
• View a topic (or all open topics) at a specific point in history
• Switch to or create a branch
• Merge one branch into another, again somehow handling conflicts
• Delete a branch

Merge conflicts automatically put Git into a changed-but-not-committed state with added markup in the affected files. Given the ability to view and edit multiple topics at once, RriegsWiki could open them all for editing with modified "Save All" and "Cancel All" buttons. Editing other topics would likely need to be disabled until the merge is resolved or canceled.

It would also be neat to support git blame and the ability to regress through history while watching the annotated content evolve.

[rriegs]

Templating and transclusion

First, to clarify the terminology:

• Transclusion is the insertion of one topic's text into the text of another topic.
• Templating is transclusion with parameters given to modify the inserted text.

Transclusion is easy enough to implement, but I'm not sure it's entirely appropriate for a single-page wiki, where it's possible to link instead and then open said link for simultaneous viewing with the parent topic. RriegsWiki could perhaps model (simple) transclusion as links that open child topics as blocks within the parents' div. This enforces a degree of logical arrangement on the child topics while permitting the parent topic to stay visually small. MediaWiki's mobile site layout does something like this with section headings.

Templating is useful for the inclusion of regularly formatted info boxes, nav bars, and other reusable content structures. In the spirit of Markdown being entirely plain-text readable, I would prefer templates (if I even support them) to add only utility and formatting, not content. The problem with templates, however, is that they typically require a more expressive specification language than the rest of the wiki, which I am hesitant to invent. MediaWiki uses a combination of "magic words", "parser functions", and Lua scripting, while TIddlyWiki has its own tangle of syntax for macros, widgets, and JavaScript modules.

Introducing user-modifiable JavaScript or Lua is at best risky, and developing my own set of built-in templates seems to invite some of the same criticisms I have against TiddlyWiki. I observe that, just as transclusion can be replaced by links in a single-page wiki, nav bars can be replaced by tags and regularly structured info boxes might as well be formatted in-line for easier plain-text reading. There's still something to be said for DRY practices, but I'm not sure they're worth the added complexity.

I should note that Hercule is a transclusion system aimed at Markdown, though it appears to be a strictly server-side. Regardless, I don't like how its transclusion and parameter substitution syntax overlap; this to me seems like a missed opportunity to have overloaded the reference-style link syntax.