denoland / deno-gfm

Server-side GitHub Flavored Markdown rendering for Deno
https://jsr.io/@deno/gfm
MIT License
221 stars 33 forks source link

Switch to a CommonMark compliant parser #54

Open kidonng opened 1 year ago

kidonng commented 1 year ago

Marked is not CommonMark compliant. It has improved significantly since I last looked at it, but there are still a few edge cases.

I suggest switching to a CommonMark compliant parser if deno-gfm is aiming for feature parity with GitHub.

There are a few choices:

I won't dive deep here because remark people has written awesome comparisons.

Personally I'm strongly in favor of remark, it makes future development easier than the other choices.

lino-levan commented 1 year ago

I think that if we move to anything, remark-gfm seems to make the most sense. We should think about how this change would interact with #55. Are there any regressions that we would expect switching to remark-gfm (outside of it being less battle-tested?)

kidonng commented 1 year ago

We should think about how this change would interact with #55.

This would/should go before any other changes I proposed, as I intended it to be.

Are there any regressions that we would expect switching to remark-gfm (outside of it being less battle-tested?)

It is battle-tested. Any regression would just be mismatching current code.

lino-levan commented 1 year ago

This seems like a good step forward, especially if we get footnote support "for free" along with the change. Are you interested in making this PR or should I do it? This should cleanup our codebase quite a bit given that remark has the remark-gfm plugin as well as remark-math.

This would also help close these at deno_blog

lino-levan commented 1 year ago

Could also close https://github.com/denoland/deno-gfm/issues/51

kidonng commented 1 year ago

It seems I'm less familiar with the current state of this/related projects than you, I would appreciate it if you are going for it.

hashrock commented 1 year ago

I'm interested in it. Does remark work without npm specifier in deno? Has anyone tried it?

lino-levan commented 1 year ago
image

It seems to just work out of the box with no crazy configuration needed. This results in:

<p><em>hello there</em></p>
hashrock commented 1 year ago

Looks great!

However, there may be an option to use it directly without a wrapper. Especially when customizability is important, such as dotland/x

lino-levan commented 1 year ago

I was playing around with remark and its family of plugins and there are some subtle issues that make it pretty hard to switch easily. The biggest one is there is somehow no syntax highlighting plugin that uses prism (that doesn't use node:vm) and we can't really switch to alternatives (since it looks like #67 has stalled). We could write this ourselves.

lino-levan commented 1 year ago

@hashrock I see you've been playing around with remark. Any luck on syntax highlighting?

hashrock commented 1 year ago

@lino-levan I haven't tried prism, but rehype-highlight (lowlight) worked. (starrynight didn't work though)

I'd like to see the impact on speed, bundle size, and rendering fidelity of GitHub's README markdown. https://github.com/hashrock/simulate-github-gfm

The rendering of katex is a little strange at the moment. I think maybe I'm using it wrong.

lino-levan commented 1 year ago

If I recall correctly, I was getting a similar issue last I checked. Would moving to lowlight be fine? I don't know how important syntax highlighting is outside of it existing.

hashrock commented 1 year ago

Hmm, I'm not sure yet. I think there is no choice but to try rendering the actual GitHub README and accumulate evidence.

Another way is to create a separate library that wraps remark. It allows things to move faster and allows projects like deno_blog to choose libraries at their own discretion.

hashrock commented 1 year ago

Study

A comparison of remark and marked can be found here:

https://github.com/micromark/micromark#comparison

If you have markdown you trust and want to turn it into HTML without a fuss, and don’t care about perfect compatibility with CommonMark or GFM, but do appreciate a small bundle size and stability, use marked

so I checked the bundle size with a script that has roughly the same functionality.

https://github.com/hashrock/simulate-github-gfm/blob/main/simple-remark.ts

% deno info simple-marked.ts
dependencies: 64 unique
size: 1.37MB
% deno info simple-remark.ts
dependencies: 252 unique
size: 1.4MB

There isn't much difference in size, but the number of dependencies has increased significantly. I'm thinking that it may affect the initial cache time and deployment time, but the time measured by deno cache -r varies considerably and I couldn't measure it well.

lino-levan commented 1 year ago

Interesting. Thanks for the notes!