Proposal: Split MDX spec into two parts: markdown extensions (mdx) and mdx-js

Proposal

Currently, as far as I've understood, markdown (i.e. commonmark) does not have a syntax for custom extensions. MDX could be a solution for this.

Actually, as it already stands, mdx acts in it's current form as a way to do exactly this. It allows extending markdown documents with a third type of syntax (alongside md and html syntax): user provided custom elements.

Simultaneously, here lies the problem: while mdx specification provides means for extending markdown documents, it is currently heavily tied to the javascript world, which is against the language agnostic nature of markdown itself.

I'm proposing splitting mdx specification in two parts: top level layer providing means for markdown extensions (mdx) in language agnostic ways, and a second level layer for language specific specification of implementing this in js world (mdx-js).

There might be synergy in cooperation with CommonMark specification for the language agnostic part of this proposed mdx split.

Use cases for mdx and mdx-js

There are several distinct use cases for proposed mdx and mdx-js. Below are two examples.

Using mdx-js (the way mdx works as it currently stands)

Current form of mdx (mdx-js in this proposal) is great for adding components into markdown documents in js environment, where you want the output to be html documents with a framework using jsx. This is basically the same use cases that exist for the current form of mdx: gatsby, mdx-deck et al. Great stuff.

Using mdx (regardless of the render target)

But what if we could do more with mdx style extensions?

What about a situation where you want to output the markdown document into multiple different enviroments, and where not all of those enviroments use javascript.

For example, such use case could be a wysiwyg editor for writing books in markdown. In the editor's user visible side, you would want to show the user mdx elements as react (or vue or whatnot) components that allow editing the component configurations visually in one view (and probably as regular text in another, a bit similar to write/preview views in github issue editor). Perhaps you would even have a third view for the actual preview.

But on the backend (this could be a server or desktop app) for the end product, you want to process the markdown document in some other language, and the mdx extension elements would act as ways to pick up the content from the extended element and convert it into your target language.

For example, you could have <SideNote> element, that in editor would show as a configuration panel for the sidenote, and in a preview would show what it looks in the final product. But during the processing on the backend, it would be transformed into some custom code depicting a sidenote element in your book making render target (whether it is pdf, epub, html, or something else). Maybe the element could also act similar to react-helmet, and do meta changes to the document.

A notable thing here is, in the examples above <SideNote> custom element is represented by three or more different implementations. The first one is in the wysiwyg view (in js), where it's a configuration component. The second is in preview view (also in js), where it is a preview component. Third is not anymore in js (probably, but naturally it could as well be also in js), but in some other backend language, where the document is processed into the final target representation. Perhaps there are even more implementations of the mdx element, for different target representations (pdf, epub, html, etc).

Problems

Currently mdx is heavily tied in with js, and on the other hand, mdx component implementations used are tied to the markdown document.

In order for it to provide means for universal use of extending markdown, split into mdx and mdx-js would be needed.

Problem 1 - use of js

There would need to be ways for doing common js things, such as providing custom elements in mdx document, without using js inside the markdown document. Perhaps this could happen through front matter or similar, or through settings or parser arguments.

Language specific implementations and specifications (e.g. mdx-js) could still allow using a specific language inside the mdx document, like is currently done in mdx. Perhaps this would be related to #17.

Problem 2 - implementation of mdx component should not be tied to the md document

As noted in the examples above, implementation of mdx element is not always same for all of the processings of the same mdx document, not even in the js world.

So there should be other means of resolution for the implementing element matching the mdx element, than declaring it in the md document in a hard coded manner.

(My use of mdx, so far, has been rather limited, so maybe this part is already possible.)

Why MDX would be great for use in extensions

As it currently works, mdx elements in an mdx document already, on a higher level, act as extensions for the markdown content. As it is, they just do that limited to js world.
The syntax of custom mdx elements differs greatly from regular markdown syntax, so it would be exceptional syntax for custom extensions, as it would provide that there would be no syntax clashes between extensions and the markdown specification (i.e. commonmark). It would avoid the problem that for example Typescript creates on the EcmaScript standard. Typescript effectively sets constraints for future es syntax due to its de facto status as typed ES, even though it's not part of the actual EcmaScript standard.

TL;DR

You guys are doing something great with MDX as it currently stands. I'm just throwing here the idea, that MDX could be even more. Instead of just being a great tool in js world, it could open the flood gates for future-safe extensions in the greater world of markdown.

I'm opening this issue here, as you guys are doing great progress with MDX. Will be opening a similar issue in CommonMark repository as well.

mdx-js / specification