💡 RFC: Components in Markdown

[FredKSchott]

What is Missing from Astro Today?

• We had to remove components-in-Markdown support from Astro before launch.
• Now, lots of us would like to add it back!
• Previous discussion: https://github.com/snowpackjs/astro/issues/364

/cc @snowpackjs/maintainers

Proposed Solution

Inspired by: https://github.com/snowpackjs/astro/issues/364#issuecomment-859573326

---
// src/pages/index.mdc
import SomeComponent from '../components/SomeComponent.jsx';
import SomeLayout from '../layouts/SomeLayout.astro';
export const metadata = {
  title: 'Title',
  description: 'Description',
  date: '12-01-1990',
  layout: SomeLayout,
};
---
# Hello, world!
<SomeComponent:visible />

But what about yaml/toml/etc?

Go for it! Use whatever parser you'd like, as long as it can return a JS object.

---
import SomeComponent from '../components/SomeComponent.jsx';
import yaml from 'yaml-parser';
export const metadata = yaml`
  title: Title
  description: Description
  date: '12-01-1990'
  layout: '../layouts/SomeLayout.jsx'
`;
---
# Hello, world!
<SomeComponent:visible />

But what about REAL yaml/toml/etc?

I think we need to support this for backwards compat with existing users, it would be a super drastic change to remove support which we should really only do in a v1.0 RC. Who knows though, maybe most users will keep using this format by default and then only changing to JS when they need imports. Kind of like defaulting to .js in your project and then only switching to .jsx when you need JSX.

// src/pages/foo.md
---
title: Title
description: Description
date: '12-01-1990'
layout: '../layouts/SomeLayout.jsx'
---
# Hello, world!
<!-- Not supported, only works with JS frontmatter: <SomeComponent:visible /> -->

Alternative Solutions

• .md.astro or .astro.md - Unnecessary, Astro syntax (ex: using components) is valid Markdown syntax.
• Astro.setMetadata() vs export const metadata - Now that we're redoing the props API to not use exports, export const metadata shouldn't conflict with anything existing.

Proposal Status

Looking for feedback, and alternative solutions. If this gets some high-level buy-in, then one of us can convert to an official RFC.

[natemoo-re]

I'm not convinced that .md.astro or .astro.md is unnecessary, at least from the perspective of VS Code.

.md files would use Code's built-in Markdown syntax highlighting, which means they wouldn't see support for things like TS frontmatter, JSX expressions, recursive Markdown, or hydration directives. Users would need to manually specify that .md files should actually use the Astro Markdown language, which is an irritating extra step and could easily trip up new users. Strictly speaking, these files aren't plain .md files. What about an extension like ~.mda~ (taken by Microsoft Access) or .mdastro if we don't like .md.astro or .astro.md?

I think I lean towards .astro.md because it is primarily a Markdown file, but it is ultimately run as an .astro file. This reminds me a lot of .html.twig files.

[natemoo-re]

I do like the idea of switching to a format more like this...

// src/pages/index.md
import SomeComponent from '../components/SomeComponent.jsx';
import SomeLayout from '../layouts/SomeLayout.astro';
export const metadata = {
  title: 'Title',
  description: 'Description',
  date: '12-01-1990',
  layout: SomeLayout,
};

But I know from user feedback that a lot of folks have tons of existing files that already use YAML frontmatter that they would want to "just work" with Astro. It would be annoying to diverge from the most common standard here.

Could we just automatically switch the parser if we detect JS keywords (import/export/const/let)? Otherwise treat the frontmatter as YAML?

Also it would be nice to have yaml and toml tagged templates built-in.

[FredKSchott]

But I know from user feedback that a lot of folks have tons of existing files that already use YAML frontmatter that they would want to "just work" with Astro. It would be annoying to diverge from the most common standard here.

Yup, agreed. Tried to think about that a bit above: "maybe most users will keep using this format by default and then only changing to JS when they need imports. Kind of like defaulting to .js in your project and then only switching to .jsx when you need JSX." Agreed that automatically detecting and switching between the two would be doable, probably easier to regex valid YAML syntax then detecting JS keywords.

Also it would be nice to have yaml and toml tagged templates built-in.

If we can find one that's 0 dependency (or very few deps) I'm game. I guess we need to keep a YAML parser anyway to keep supporting YAML in frontmatter.

[FredKSchott]

.md files would use Code's built-in Markdown syntax highlighting

Is there no way for the Astro VSCode extension to help out here? If in an astro project, default to this other syntax?

[matthewp]

Allowing JS in frontmatter fixes 2 of the issues of the previous markdown implementation. The other is syntax highlighting, as @natemoo-re mentioned. I know that users have asked for using components in markdown files, but I wonder, is that because the alternative we gave them is too low-level or is it really that they want the extension to be .md? If we didn't allow .md at all and forced them to write .astro.md or .mdastro or whatever else, would that be ok? I think the mistake we made before was leaving .md in and markdown-only. I think we should ask the community about this, is it really important to them that the extension to be .md?

The other thing about this proposal is, I don't think we should deprecate yaml in markdown. Asking people to write JavaScript for a blog post that is mostly textual is asking a lot imo. If we forced JS I think we would be skewing too heavily towards the "writing blog posts with coding examples" use-case which is an important one but also niche in the grand scheme of websites. Also if anyone is using an external CMS it might provide markdown with yaml frontmatter, so I think we'll need to support that either way.

I don't see why we could take an approach of try/catch the frontmatter as yaml and if it doesn't parse treat it as JS. This would not be a performance issue since it's dev only. We could also ask them to use a directive like --- yaml or --- js but I'm not sure how they'd feel about that.

[natemoo-re]

Is there no way for the Astro VSCode extension to help out here? If in an astro project, default to this other syntax?

Unfortunately not. Language settings are configured by the user. We can't really ask users to switch their configuration for .md files because of the places we diverge from CommonMark (no significance to indentation, recursion).

IMO sticking our syntax into .md feels like writing JSX in a .js file. Is it possible? Sure! But it causes more issues than it solves.

What about .mdc—Markdown Component files?

[FredKSchott]

I don't see why we could take an approach of try/catch the frontmatter as yaml and if it doesn't parse treat it as JS. This would not be a performance issue since it's dev only. We could also ask them to use a directive like --- yaml or --- js but I'm not sure how they'd feel about that.

regex to detect yaml should be pretty straightforward! nate and I were just talking about this in the comments above. lmk if my post could be clarified to make this more clear.

[FredKSchott]

Unfortunately not. Language settings are configured by the user. We can't really ask users to switch their configuration for .md files because of the places we diverge from CommonMark (no significance to indentation, recursion).

IMO sticking our syntax into .md feels like writing JSX in a .js file. Is it possible? Sure! But it causes more issues than it solves.

That's fair enough. Not 100% convinced BUT regardless of how I feel, if its not possible to get syntax highlighted correctly then I'm a -1 as well.

What about .mdc—Markdown Component files?

Haha well that's actually pretty slick. I'd get on board with this. Who says that MDX is the only game in town? 2021 is the year of MDC :)

[FredKSchott]

I think we should ask the community about this, is it really important to them that the extension to be .md?

I posted this in the #dev channel already, and we should revisit during next weeks RFC call.

[jasikpark]

I'm a fan of .astro.md to signify that it's not vanilla markdown!

[FredKSchott]

One reason I'm anti .astro.md is that I'm worried about what happens when React lands .server.js and .client.js server components, would having .astro.md feel strange then? Maybe that's an overreaction though! If everyone else likes it, then I'm definitely not blocking it :)

[FredKSchott]

Alternative designs:

---
export default {
  title: 'Title',
  date: '12-01-1990',
};
---

---
export const frontmatter = {
  title: 'Title',
  date: '12-01-1990',
};
---

Not a dealbreaker, but would like to have layout be a runtime variable via ESM import (is currently a string in Astro)

[georges-gomes]

Pushing some ideas here.

1) Open .mdc (from @natemoo-re )

There is an opportunity to make .mdc an open format - not specific to astro.

---
import SomeComponent from '../components/SomeComponent.jsx';
export cont title = 'foo';
---
...

Nothing specific to the underlying implementation.

Something as simple as that.

astro can have helper methods to load .mdc as astro components.

2) Keep .md and .mdc separated

I think you could keep .md with yaml and toml frontmatters. and treat .mdc separately for js front-matter and component capabilities.

astro code already has ConverMDtoAstro(), would not be hard to have ConvertMDCtoAstro() on the side of it.

Would help people migrate to Astro without rewriting frontmatters and creates a clear separation of the 2 formats.

(+ VS Code extension easier I think)

3) Should we treat .md and .mdc has pages?

I'm wondering if treating these files as pages is a problem. Because they are loaded as data from the Collection API.

I think .astro files should be the only routing setup and all the rest is data. This would probably simplify a few little corner cases on md...

My 2 cents

[natemoo-re]

We'll be moving forward with this "markdown + components" file format separate from .md, with the caveat that we need to hash out the final extension.

We also want to discuss if this is an open format or something directly tied to Astro.

[FredKSchott]

.astro & .astromd? :)

[jasikpark]

In my opinion, I think making "markdown + components" an open format makes sense, but that at the same time it should be tied to .astro as some sort of a variant, just with Markdown as the base parser rather than HTML.

[Brixy]

One more point: If I get it right components are (also) a more powerful counterpart of traditional shortcodes—used in Nunjucks, Liquid et al.—that are used to enhance Markdown.

So this would be an important functionality for anybody switching from another SSG to Astro.

[jasikpark]

Yup, that's a big reason why I'm interested in this shipping at some point!

[mash-graz]

compatibility with existing markdown frontmatters would be IMHO a really important feature.

i always had to fight this issue in the case of rust SSGs, because some people in the rust community simply hate YAML and do not want to support any frontmatter processing except for TOML. that's rather stupid and makes it unnecessary hard to switch the tools resp. evaluate and esteem all other advantages of some alternatives.

IMHO we should support all common used types of markdown frontmaters and the frequently used conventions to differentiate between the used syntax:

see: https://gohugo.io/content-management/front-matter/#front-matter-formats

Hugo supports four formats for front matter, each with their own identifying tokens:

* TOML
identified by opening and closing +++.

* YAML
identified by opening and closing ---.

* JSON
a single JSON object surrounded by ‘{’ and ‘}’, followed by a new line.

* ORG
a group of Org mode keywords in the format ‘#+KEY: VALUE’. Any line that does not start 
with #+ ends the front matter section. Keyword values can be either strings (#+KEY: VALUE)
or a whitespace separated list of strings (#+KEY[]: VALUE_1 VALUE_2).

augmenting this common frontmatter usage by just allowing additional JS markup resp. look for the relevant keywords therefore seems to be the most compatible and desirable solution for component support in markdown files by astro IMHO.

[briannhinton]

Would we still be able to fetch markdown using this approach? I know what many of us do, and want to do is have components within articles / posts / notes that we are writing. That's the big advantage of .mdx.

[natemoo-re]

@mrbrianhinton This is a really great point! Could you clarify how this works with .mdx? We should make sure to solve this in a way that addresses these issues as well.

[briannhinton]

This is how MDX works. https://mdxjs.com/advanced/components

import MyComponent from './my-component'

export const author = 'Fred Flintstone'

# Title

<MyComponent />

Lorem ipsum dolor sit amet.
MDX core turns that text into roughly the following JSX to be consumed by your app:

import React from 'react'
import MyComponent from './my-component'

export const author = 'Fred Flintstone'

const layoutProps = { author }
export default function MDXContent({ components, ...props }) => (
  <wrapper {...props} {...layoutProps}>
    <h1>Title</h1>
    <MyComponent />
    <p>Lorem ipsum dolor sit amet.</p>
  </wrapper>
)

[mash-graz]

i think it's really important, that we use the correct/common "fences" for the different kinds of frontmatter content!

as long as it's only containing simple YAML, the --- notation works well, but we shouldn't put arbitrary JS code in this kind of frontmatter without changing the fences/delimiters as well or wrap the code in a multiline YAML string field etc., otherwise it will break the correct rendering of the affected markdown files in other applications and web-based git-hosting-platforms.

i really understand, that it looks desireable to have the same frontmatter sytax in .astro and .md files, but nevertheless we also should respect the expectations of other software resp. general compatibility issues.

[jasikpark]

An interesting other implementation of "components in markdown" esque function is https://github.com/remarkjs/remark-directive which adds shortcodes to markdown.. I don't think the redirection of syntax is worth the effort though...

[JSHSJ]

I'd be fine with either the mdx approach or the remark-directive approach.

The latter would need some sort of config, where you would define the components to use for each directive, but would adhere to remark standards and would hypothetically allow us to move our .md files between different renderers (Astro and someplace else), which is always a good thing in my book.

The most important feature for me is an easy setup and the ability to use my standard markdown files (i.e. .md), because I'd like to be able to edit / manage them in different places (like Obsidian or even a CMS like forestry) without having to edit my build actions to rename all the files to a new syntax.

[brycewray]

Just a note from the already-using-other-SSGs-but-also-digging-Astro peanut gallery: am very much looking forward to this capability, however you implement it, specifically for what @Brixy said earlier:

One more point: If I get it right components are (also) a more powerful counterpart of traditional shortcodes—used in Nunjucks, Liquid et al.—that are used to enhance Markdown.

So this would be an important functionality for anybody switching from another SSG to Astro.

I just finished converting my Eleventy image-processing shortcode to JSX for exactly that reason. While I'll implement in an .astro file using <Markdown> for now while I'm just playing with this, would be ultra-cool to be able to stick with the vast majority of my existing Markdown files' content and simply insert components! (I detest Gatsby with a passion, but can't deny the coolness of MDX for this sort of thing.)

Thanks for your efforts!

[jasikpark]

I feel like an interesting option would be to make the Markdown component really robust and just add collections for astro pages in addition to markdown pages

[rebelchris]

I was wondering if there was any news on this issue. I would really love to have an option to embed components in some way.

If it means having markdown parsers that's also cool, I think that's how a lot of different platforms solve it for now. EG:

# Heading 1

{{ tweet 121212 }}

or: 

{% tweet 1212 %}

whatever is unique enough. Still hoping we can however somehow make astro components work 🗡️

# Best case:

<Tweet id="132332" />

[Porges]

JSX-style components really need to be able to work in order to pass structured (read: JS object) data to components.

e.g. I have a special image component whose usage looks like:

<ArticleImage
    src={imgYoshino}
    position="aside"
    size="wide"
    alt="A mountain covered in cherry blossom trees showing light pink blooms
         amongst other dark green trees."
    source={{
        license: 'cc-by-nc-nd',
        licenseVersion: '4.0',
        originalUrl: 'http://photozou.jp/photo/show/314766/35962820',
        copyrightYear: 2010,
        author: { name: 'ゆぼ', lang: "ja" }
    }}>
    The <Noun lang="ja-Latn">Yoshino</Noun> mountainside with cherry trees in bloom.
</ArticleImage>

[Porges]

However, overall #1142 seems easier to achieve than this!

[flikteoh]

It would be nice if markdown files can remain markdown files, since not all markdown users are developers.

Would like to suggest instead:

Show code

```javascript --- // src/layouts/Posts.astro // any or all framework components import AstroComponents from '../components/astro/'; import PreactComponents from '../components/preact/'; import SvelteComponents from '../components/svelte/'; import VueComponents from '../components/vue/'; import SolidComponents from '../components/solid/'; import Cats from '../components/solid/Cats/'; // or custom markdown mapping import Heading from '../components/preact/Heading/'; import CustomLink from '../components/preact/CustomLink/'; import Image from '../components/preact/Image/'; const CustomMapping = { h1: (props) => , a: (props) => , img: (props) => , } import BaseHead from '../components/BaseHead'; import BlogHeader from '../components/BlogHeader'; import BlogPost from '../components/BlogPost'; import { Markdown } from 'astro/components'; const { content } = Astro.props; const { title, description, publishDate, author, heroImage, permalink, alt } = content; --- ``` Since we already allow the use of `` to parse external contents, can we also parse custom components during that? Example: If the post/page content from `.md` files doesn't use the ``, then we just don't render for those posts.

Show code

```javascript --- // src/layouts/Events.astro // Use case: Different page layouts, seasonal component decorations, etc. // Default Preact components switched out for event day for example // import PreactComponents from '../components/preact/'; import EventDayComponents from '../components/solid/EventDayComponents/'; // or custom markdown mapping import Heading from '../components/preact/Heading'; import CustomLink from '../components/preact/CustomLink'; import Image from '../components/preact/Image/'; const CustomMapping = { h1: (props) => , a: (props) => , img: (props) => , } import BaseHead from '../components/BaseHead'; import BlogHeader from '../components/BlogHeader'; import BlogPost from '../components/BlogPost'; import { Markdown } from 'astro/components'; const { content } = Astro.props; const { title, description, publishDate, author, heroImage, permalink, alt } = content; --- ```

This way, we could prevent confusing markdown users who aren't developers.

For example:

• having to import multiple components for every posts
• using components they forget to import on every posts
• relative links to components in a nested folder structures?

For scenarios like relative links, I think the Astro Example Portfolio also makes a nice case on the layout: frontmatter between nested posts.

We can make use of dynamic routes to apply layouts or different layouts on a developers level instead.

While developers gets more granular controls on what custom components to provide on each post-types without having to introduce a new extensions for markdown files in Astro.

Further example:

// inside some markdown files
---
title: My Post title
slug: /post/
setup: |
  import Component from "../../../../components/Component";
---

## My Heading

<Component client:visible />

My content

A non-developer markdown user may not understand the usage of client:visible in this case. It would be more ideal if the developer can decide at component level for example.

This approach also less likely to break things on editors or plugins:

I hope the above can be taken into considerations.

While I understand generally everyone has different use cases, but retaining markdown files as they are would likely be easier to manage since frontmatter aren't tied to required fields. It is also easier for developers to make changes without having to regex all markdown files to fix or correct things.

p/s: I'm looking forward to move a site over to Astro but there are nested content folders, group of users who are non-developers, and over 1,000 markdown files to import multiple components/adding layout frontmatter into.

[dbowling]

I’d agree that existing file extensions like .md should keep compatibility expectations managed. For this reason I feel that MDX is a really solid fit for this problem, and really looks like the initial proposal, which makes a very usable and familiar alternative to the current Frontmatter. I saw Remark mentioned above, which also is in the MDX implementation guides.

Is there any barrier to MDX in Astro—it seems like the most robust and aligned solution—or is it still mostly a matter of landing on consensus regarding the various options?

[tstachl]

I just recently discovered this feature has been implemented in astro@0.21.0-next.3 like this:

---
title: My Video Post
setup: |
  import VideoComponent from "../../components/Component";
---

## Watch the Video

<VideoComponent videoId="1234323" />

My content

[FredKSchott]

correct, this will ship in v0.21!

[Enteleform]

Just tried out the

---md
setup: |
  import ...
---

feature & it's working great!

Wondering if there's a mechanism like mdx-js & xdm's components attribute that allows you to use components without directly importing them in md files.

e.g. @ index.astro

---
import Article       from "./Article.md"
import SomeComponent from "./SomeComponent"
---

<Article components={{SomeComponent}}/>