MDXv2 broken with remark-heading-id

Eyas commented 2 years ago

Initial checklist

[X] I read the support docs
[X] I read the contributing guide
[X] I agree to follow the code of conduct
[X] I searched issues and couldn’t find anything (or linked relevant results below)

Affected packages and versions

2.0.0

Link to runnable example

No response

Steps to reproduce

Install & import remark-heading-id
in MDX remarkPlugins, add remark-heading-id
Try to parse MDX: ## foo {#heading}

Expected behavior

The plugin handles the {#...} extended markup syntax, and MDX parses the document normally

Actual behavior

Get error: Could not parse expression with acorn: Unexpected character '#'

It makes sense that MDX thinks that {...} syntax is inline JS, but I would have thought plugins can be used to support heading ID Markdown extended syntax.

Runtime

Node v16

Package manager

npm v8

OS

Windows

Build and bundle tools

No response

ChristianMurphy commented 2 years ago

https://github.com/imcuttle/remark-heading-id hasn't been updated in several years, and doesn't support the latest version of remark/micromark (https://github.com/imcuttle/remark-heading-id/issues/1) which by extension means it doesn't work with the latest version of mdx. This can be resolved by opening a PR to remark-heading-id upgrading it to support the new and improved micromark parser.

Eyas commented 2 years ago

Hm. I have a local version of remark-heading-id that uses "unist-util-visit": "^4.1.0" instead (which should use micromark afaict?) and it doesn't seem to work still. Am I missing something?

ChristianMurphy commented 2 years ago

There's a deeper upgrade needed to make it work with MDX. The current approach, using unist-util-visit works on the AST, after the document has already parsed, which means JSX parsing will always take precedence and cause an issue before the plugin can do anything. If you want to add a new syntax feature, it needs to be implemented as a part of the parser. See https://github.com/micromark/micromark#extending-markdown and https://github.com/micromark/micromark#creating-a-micromark-extension for more information.

Eyas commented 2 years ago

Thanks, that helps! One more question: I'm looking at @mdx-js/mdx's CompileOptions -- where would a micromark syntax extension be registered? I assume it's not a regular remark Plugin in this case, right?

ChristianMurphy commented 2 years ago

It would be registered as a plugin, similar to how MDX itself registers itself and extends remark and micromark https://github.com/mdx-js/mdx/blob/e527ac758c5d374289d485dfb8655babb4c7c261/packages/remark-mdx/index.js#L10-L39

Eyas commented 2 years ago

Thanks for all the pointers! ended up making a separate package: remark-custom-heading-id. The micromark code is substantially different than the unist-util-visit-based approach, and the old extension is still good-enough for non-MDX cases.

rikusen0335 commented 1 year ago

I'd say thank you so much @Eyas that repo helped me a lot.

Danii2020 commented 1 year ago

Hello, hope you can still help me, I want to use the remark-custom-heading-id to parse custom heading IDs, I'm using the mdxToMarkdown to allow MDX expressions but that function doesn't allow me to use the custom heading id syntax {#custom-id}. I'm using the following function to convert to MDX:

function mdxStringify(this: Processor) {
  const compiler = (tree: Node): string => {
    const markdown = toMarkdown(tree, {
      extensions: [gfmToMarkdown(), mdxToMarkdown()],
      bullet: "-",
      emphasis: "_",
      fences: true,
      resourceLink: true,
    });
    return markdown;
  };

  Object.assign(this, { Compiler: compiler });
}

If I remove the mdxToMarkdown function it handles custom heading ids properly, but I need that function, so is there a way to add a the remark-custom-heading-id as a plugin in that function? Thanks in advance.

ChristianMurphy commented 1 year ago

@Danii2020 welcome! 👋 You don't appear to be using MDX directly? Also you asked about parsing, but shared your compiler code 🤔 And your question appears to be more related to https://github.com/Eyas/md-heading-id?

Some suggestions on how to proceed:

Your question appears to go beyond this issue, consider opening a question at https://github.com/Eyas/md-heading-id for how to integrate it into your workflow
If you do want to ask the broader MDX community, considering opening a discussion https://github.com/orgs/mdx-js/discussions
Either way, take some additional time framing the question https://mdxjs.com/community/support/ with a focus on what you want to achieve and why
And consider sharing a runnable example in a codesandbox or stackblitz sandbox.

mdx-js / mdx