Markdoc support

[bholmesdev]

This RFC introduces Markdoc support to Astro Content Collections. Resolves #496

Links

• Fully rendered proposal
• Stage 2 proposal
• Stage 1 proposal

[bholmesdev]

Alright, excited to share our findings from the performance benchmark!

Performance Benchmark

As part of the @astrojs/markdoc integration, we built a performance benchmark to compare Markdoc build performance at scale against our existing content formats, Markdown and MDX.

Method

This suite generates content collection entries across 3 separate test fixtures (one for each file type). This will be repeated across 3 different test templates at both 1000 and 3000 generated posts:

• Simple case (md, mdx, mdoc): includes 1 frontmatter entry, a heading, and plain Lorem Ipsum text content. This should compare text rendering speed.
• Astro components case (mdx, mdoc): this includes the same frontmatter as the simple case, plus 2 examples for rendering Astro components. The first example maps HTML elements (the h1 tag) to an Astro component. This use the components Content prop available to both Markdoc and MDX. The second example imports and renders an Astro component in the file, using ESM imports for MDX and tags + necessary transform config for Markdoc.
• React components case (mdx, mdoc): this also includes the same frontmatter as the simple case, plus both a server-rendered and hydrated React component (a simple LikeButton). This uses ESM imports for MDX and tags + necessary transform config for Markdoc.

Each fixture is built using the astro build command. We have also included an environment flag to disable nonessential remark plugins for Markdown and MDX, including GFM, Smartypants, heading ID generation, and Shiki syntax highlighting. Since our Markdoc integration does not (yet) include these features, this should ensure an even playing field for base render performance.

Results

Below are the overall build times for each file type across each test template, with times averaged across 3 separate test runs:

1000 generated posts

	Simple	With Astro components	With React components
md	17.52s	N/A	N/A
mdx	18.17s	18.27s	18.73s
mdoc	11.03s	7.89s	6.99s

3000 generated posts

	Simple	With Astro components	With React components
md	55.31s	N/A	N/A
mdx	48.47s	133.24s ⚠️ CPU ceiling	133.96s ⚠️ CPU ceiling
mdoc	30.66s	23.27s	24.56s

⚠️ CPU ceiling - CPU usage hit 100%, throttling builds

Discussion

There are several promising findings from this test. First, Markdoc was noticeably faster than Markdown and MDX across all test cases. This could partially be due to the speed of Markdoc's bespoke transform pipeline compared to remark and rehype. There is also a difference in rendering strategies. While MDX compiles to JSX directly, requiring a pass through Astro's JSX runtime to render as JavaScript, Markdoc compiles to a serializable JSON tree we can render directly at template build or SSR runtime. This process proves to be less CPU intensive, with MDX hitting a CPU ceiling at 3000 posts on our test machine.

Aside: This also means the @markdoc/markdoc package will be shipped to your SSR runtime, which can prohibitively large (40kb compressed) for edge compute.

Second, we noticed that Markdoc is slowest during the simple test case, while Markdown and MDX see a performance hit when rendering components. This means, unlike Markdown and MDX, Markdoc is slowed less by complexity of elements and more by the volume of HTML elements rendered (Lorem Ipsum paragraphs rendered in the simple case). It's worth analyzing whether our own render stage or Markdoc's transform stage is to blame here.

Future exploration

There are several areas we'd like to explore to further compare Markdoc against our existing MDX + remark pipeline:

• How do Markdoc transforms compare to remark plugins? This test operates without any Markdoc transformation overrides. What will be the % change in build times adding, say, a Shiki configuration to Markdoc? How does the % change compare to our MDX rehype plugin today?
• How does Markdoc compare on memory usage? This test only compares build times. We're curious if Markdoc can address the spike in memory usage MDX experiences in our prebuild step.
• How does a large library of tags compare to a large library of components? This test uses just one or two components across test files. Once component libraries scale into the dozens or even hundreds, how will Markdoc compare?

[ancaemcken]

@bholmesdev @flexdinesh @FredKSchott I want to point out that, in keeping .mdoc, Astro is locking out Markdoc users from pretty much any git-based Markdown CMS - for example FrontMatter - haven't tested Tina or Netlify CMS yet, and even some popular Markdown editors (Obsidian, Typora, etc.).

Sure, a writer could fiddle with default applications for opening a given file type, but even so, .mdoc files might be treated as second-class citizens. Such an example is Obsidian, where you can add .mdoc to the list of supported files with a plugin. Still, the support is limited to opening and editing the file, missing out on all other features for which an author chooses Obsidian - because of .mdoc.

Also, not everyone writing Markdoc is a developer or fond of using VS Code - so far, the only tool working with .mdoc without any fuss.

LATER EDIT: See https://github.com/withastro/astro/pull/6847. All tests are passing.

I made it so .mdoc.md files would be ignored by the normal Markdown processing, but handled by the Markdoc integration. This is helpful to solve editor issues, while still allowing users to distinguish Markdoc from Markdown files.

If you try the markdoc example, you can see a "plain" markdown example at http://localhost:3000/plain, whereas the index page shows the Markdoc file (now renamed to intro.mdoc.md).

[matthewp]

Closing as Markdoc support is now stable in Astro. We are going to create a new RFC when/if Markdoc comes into core, but it can continue to evolve as an integration for now.