Support extending parser to handle wiki-style links

[CPColin]

The module will be useful for displaying Ceylon documentation only if the Parser can handle wiki-style links that are delimited with double brackets. Rather than add such support to the code that was ported from commonmark.js, we should see if we can use the custom_inline or custom_block node types here. If we can't, we should see if we can extend the parser and renderers in some other way.

Another possibility is to implement this change via the renderer only. That is, if the parser produces a link node whose destination begins and ends with brackets, the renderer could (attempt to) treat it as a wiki link. This strategy would make sense for things like the IDE and the documentation tool, because they'd need to make their own metadata lookups and transformations to the links.

[CPColin]

Looks like we need to extend the parser, no matter what, because double brackets, on their own, don't parse into a link node.

[CPColin]

Current strategy, inspired by rjeschke/txtmark, which is what the Ceylon IDE code currently uses:

• The inline parser will check to see if it has two left brackets in a row by matching against this regex when one left bracket is found: \[\[.*?\]\]
• If the parser finds a match, a node with a new type, like "special link," will be generated. The text between the brackets will be parsed as children of the node.
• A new parser option will enable the new behavior.
• Proof-of-concept code will walk the AST and transform nodes with the new type to normal links, after resolving their destinations. (This step would happen in the Ceylon IDE code.)
• A utility function could be added to the module that handles the tree traversal logic, like transformSpecialLinks(Node root, Node?(String) transformation). That transformation function could maybe return String? and [String, String]? also. (Tinker with this and see what works best.)
• Both renderers will show the new node type as "[[" + render(node.children) + "]]". The assumption is that any "special link" nodes still in the AST were unable to be transformed.

[CPColin]

Here's an example of how one would transform special links:

import ceylon.markdown.parser {
    Node,
    NodeType,
    ParseOptions,
    Parser,
    transformSpecialLinks
}
import ceylon.markdown.renderer {
    RawHtmlRenderer
}

Node createLinkNode(String destination, String content) {
    value node = Node(NodeType.link);

    node.destination = destination;

    value text = Node(NodeType.text);

    text.literal = content;

    node.appendChild(text);

    return node;
}

shared void run() {
    value input
            = "## Testing special links
               Broken link: [[broken]], working link: [[link]].";
    value parseOptions = ParseOptions {
        specialLinks = true;
    };
    value parser = Parser(parseOptions);
    value root = parser.parse(input);
    function transform(String content)
            => content != "broken" then createLinkNode("doc:``content``", content.uppercased);

    transformSpecialLinks(root, transform);

    value renderer = RawHtmlRenderer();

    print(renderer.render(root));
}