How should I enable "shortcodes" or "text macro processing"?

[deanebarker]

We have a desire to do some type of text macro system in Wyam. These would be small snippets of text in Markdown that would "inflate" into larger constructs.

For example:

[embed-tweet /gadgetopia/status/749204741209460736]

When processed, that would generate the HTML necessary for a tweet embed. This is one example among many. In addition to singularities like the above, there are lots of opporunities for open/close pairs:

[aside title="Here's the story..." color="astroturf"]
...of a man named Brady...
[/aside]

That would generate all the markup necessary for an <aside> element, correctly styled with a title.

Note that these are just simple examples to illustrate the idea. I realize you can pick these apart ("Why not just..."), but don't get bogged down in the specifics. Just assume there's a general, valid need for "text macro processing."

In fact, there are lots precedents here. Here are three...

• WordPress has had shortcodes for a while
• Drupal has the Token Filter module which enables the same general thing
• Hugo also has shortcodes

(In searching for this information, I'm starting to think that "shortcodes" is becoming a neologism for this type of functionality. I don't know if it's totally accepted yet, but this is what people are calling this.)

I've been playing around with some ideas. For example:

• I used Markdown.EscapeAt(false) to retain the @ symbol and therefore just enabled the direct insertion of calls to Html.Partial in Markdown
• I did some text parsing and processing to replace [[ embed-tweet ...]] with the correct partial inserton code
• A considered pre- or post- processing the Markdown with something like DotLiquid which has lots of support for this

However, all of these are not great for one reason or another. So, I have two questions:

1. Has someone done this before? Is there prior art I'm just not seeing? Are these Snippets?
2. Jurisdictionally, where does this fall? Is this something I should do in (1) a Wyam module; (2) Markdown, as a Markdig extension, or (3) Razor?

[LokiMidgard]

I had done something similar in the past. But I find my approach very hacky. I used Markdown ::: Syntax to generate divs with some additional information and the page contains javaScript that searches for those classes and manipulates them untill they look like something I need.

I used this for some expander with headers and body.

[daveaglick]

First, some answers:

Has someone done this before?

Not that I'm aware of - at least not in a general sense, though I know lots of folks have done one-off text replacement hacks for specific functionality like @LokiMidgard

Is there prior art I'm just not seeing?

Nope, don't think you missed anything

Are these Snippets?

Yes, exactly. Now that 2.x is released and I'm working through the bug backlog, this is on my shortlist for what to work on next. There are a lot of feature requests blocked by something like this so I'd like to get the foundation in place soon. I'm still pretty fond of the designed scoped out in https://github.com/Wyamio/Wyam/issues/359#issuecomment-266906339 with the additional caveats in the notes that follow it (I'll reword the design to incorporate those comments).

As an aside, your point about "shortcodes" becoming an accepted and well-understood term is a good one - I didn't want to call it that initially for fear of confusion with Hugo, but since it's becoming a more general term maybe it's time to rethink that. And "snippets" seems to have another conventional meaning related to code snippets (which is captured in issue #692).

Jurisdictionally, where does this fall?

I think the right way to do it is independent of any particular rendering engine as a post-processing step. Wyam only supports Markdown and Razor out of the box right now, but I absolutely intend to support Liquid, Scriban, and others in the future. I'd want whatever shortcode functionality we add to be orthogonal to the rendering engine.

That means it's probably "turned on" by a module that processes documents and does shortcode replacement. This could be a special module in that shortcode capabilities could be built into the core libraries and the module just applies them. I think that part of the design in #359 is still pretty solid.

---

So where does that leave us?

I'll go ahead and work on the design in #359 a bit more and now that you've nerd-snipped me, I'll probably start working on implementation. In the meantime, I'd suggest you use something like simple text replacement until the real thing is ready.

My first thought on a quick and dirty way to do this is to use the HtmlInsert module or write a new module that's very similar and uses AngleSharp to do the replacement based on the DOM in a post-processing phase. Then you could put HTML in your Markdown or Razor layouts and have the post-processing module(s) find the HTML and swap it out for what you need. I actually considered basing the more general shortcode/snippet functionality on HTML, but wanted it to be general enough to work outside of web contexts (but you don't have that problem).

In the meantime, I'll close this issue in favor of continuing to work and track shortcodes in #359.