Cannot refer to fractive macros within story text

[invicticide]

Macros are currently expanded regardless of context, so trying to do something like this:

This is some story text which would like to tell you that macros look like this: `{#Function}`

will just expand the macro, even though it's enclosed in backticks. Code blocks (backticks and indents) should suppress this behavior.

Need to add this context awareness to Core.ExpandSection().

[invicticide]

Digging into this and it turns out to need a more sophisticated solution, since one may need to refer to macros without expanding them in all sorts of contexts, not just within code blocks.

(Admittedly most authors won't care about doing this, but I really want to create an interactive tutorial for Fractive that's actually written in Fractive, for which this is super necessary).

The real solution here is to escape macros just like you'd escape anything else, so:

We'll expand this macro: {$theVariable}
But we'll just display this one: \{$theVariable}

...results in:

We'll expand this macro: 42
But we'll just display this one: {$theVariable}

The escaping rules in Markdown are slightly tricky. Normally, any combination of \x for any value of x is resolved as an escape sequence, but if it has no recognized meaning, commonmark just drops the backslash and we just get the x. To ensure it comes through to us as \x you actually have to double-escape it in the source text, like \\x. However, if you're in a code element (inline, block, or fence) you only want to single-escape, like \x. (That's all Markdown, not us.)

Once we get the right source data into Fractive, we would've parsed for escape sequences in the compiler (for links) and again in core (for expanding sections). That's dumb, and plus, doing all this after-the-fact text parsing is brittle. So branch issue/8 (WIP) rewrites the compiler to use the AST instead.

Now, we get the AST back from commonmark and then scan through that looking for nodes with text content. We parse that content -- which is still in Markdown syntax, not yet rendered to HTML -- for macros and escaped macros. Real macros get extracted and turned into a new AST nodes (we insert an html_inline which resolves to a <span> with a data attribute indicating the type and target of the macro to expand). Escaped macros are left as-is. (This AST pass also rewrites links a similar way.) The end result is that our final HTML contains no functional macros at all; instead, it contains <span> nodes with functional data attributes. And that means core can just walk the DOM and respond to those attributes, never needing to parse any text at all.

As of f3a8f73 I have the compiler part of this hooked up, but not the core part (that's next). There are also a few element types that aren't supported yet (e.g. using a macro as an <img> source URL).

[invicticide]

The core part is hooked up and we're back to fully functional as of 7c12053. However, the issue/8 branch remains unmerged, as there are still several outstanding TODOs:

• Macros as img urls don't work yet (there's an image node type)
• Fix up sourcepos for error reports in compiler
• Test error reports in compiler
• Split up the giant switch/case in RenderFile
• Set link alt text to the macro url for easier debugging in browser
• Check other node types for handling: block_quote, item, list, heading, html_inline, html_block, thematic_break

[invicticide]

As of a43f93b this issue is basically resolved. There are still a few things from that TODO list in my previous comment but they're really separate issues at this point.