remarkjs / react-markdown

Markdown component for React
https://remarkjs.github.io/react-markdown/
MIT License
13.27k stars 876 forks source link

use plugin rehype-highlight v7.0.0 memory leak #791

Closed zzzgydi closed 2 months ago

zzzgydi commented 1 year ago

Initial checklist

Affected packages and versions

rehype-highlight v7.0.0

Link to runnable example

https://github.com/zzzgydi/rhv7-memory-leak-test

Steps to reproduce

As a comparison

Expected behavior

same as rehype-highlight v6.0.0

Actual behavior

image

Runtime

No response

Package manager

pnpm

OS

macOS

Build and bundle tools

Vite

immccn123 commented 1 year ago

can reproduce in https://remarkjs.github.io/react-markdown/

just type freely in the editor and a large increase of memory will be found

ChristianMurphy commented 10 months ago

Thanks @zzzgydi and @immccn123! Memory should not be ballooning in that way. Would either of you be interested in tracing this further and potentially opening a PR? https://developer.chrome.com/docs/devtools/memory-problems

wooorm commented 10 months ago

I’m pretty sure V8 and friends actually use up as much memory as they can before freeing it? Maybe you’re seeing memory being used?

zzzgydi commented 10 months ago

I guess that lowlight has registered languages internally, but did not unregisterLanguage. After creating multiple instances like this may cause memory leaks.

Below I have written three comparative examples. In my understanding, if there is no memory leak, the memory should be released after 100 iterations of the for loop.

import { createLowlight, common } from "lowlight"; // version: ^3.1.0
import HighlightJs from "highlight.js"; // version: ^11.9.0
import "./App.css";

function customCreateLowlight(grammars: typeof common) {
  const high = HighlightJs.newInstance();
  if (grammars) {
    Object.keys(grammars).forEach((name) =>
      high.registerLanguage(name, grammars[name])
    );
  }
  return high;
}

function App() {
  return (
    <div style={{ display: "flex", flexDirection: "column", gap: 12 }}>
      <button
        type="button"
        onClick={() => {
          for (let i = 0; i < 100; i++) {
            const lowlight = createLowlight(common);
          }
        }}
      >
        createLowlight
      </button>

      <button
        type="button"
        onClick={() => {
          for (let i = 0; i < 100; i++) {
            const lowlight = customCreateLowlight(common);
          }
        }}
      >
        customCreateLowlight
      </button>

      <div>
        <button
          type="button"
          onClick={() => {
            for (let i = 0; i < 100; i++) {
              const lowlight = customCreateLowlight(common);
              Object.keys(common).forEach((name) =>
                lowlight.unregisterLanguage(name)
              );
            }
          }}
        >
          customCreateLowlight & unregisterLanguage
        </button>
      </div>
    </div>
  );
}
export default App;

I obtained the following results. The results are divided into three rounds. First, I refresh the page, recorded, and then click on the button to run once, and recorded next.

image image

Through the above graph, it is a comparison of the results between snapshot 6 and snapshot 5. The largest occupation is string type, and these values are related to language settings within highlightjs.

wooorm commented 10 months ago

After creating multiple instances like this may cause memory leaks.

If there is indeed a memory leak instead of chrome using up the memory it can use up, then that seems like a highlightjs issue?

It’s JavaScript; you’re not supposed to need to manage your own memory by removing registered things. The language does garbage collection for you.

TonyL1u commented 10 months ago

I met the same problem, v7 did not work well when v6 was ok. Is there any solutions for this problem?

wooorm commented 10 months ago

The solution is you can contribute to open source and figure out a solution instead of spamming duplicate info.

TonyL1u commented 10 months ago

The solution is you can contribute to open source and figure out a solution instead of spamming duplicate info.

good advice

alielmorsy commented 9 months ago

Did anyone find a solution for that? I am using that code and yes there is a weird memory leak because it renders code blocks over and over:

   <Markdown rehypePlugins={[rehypeRaw, rehypeHighlight]} components={{
                code({node, className, children, ...props}) {
                    return <div className={className}>
                        {children}
                    </div>
                },
            }}>
                {post.content}
            </Markdown>
remcohaszing commented 9 months ago

If possible, you should provide components with a stable identity to avoid rerenders.

function Code({ node, className, children, ...props }) {
  return (
    <div className={className}>
      {children}
    </div>
  )
}

export function Post({ post }) {
  return (
    <Markdown
      rehypePlugins={[rehypeRaw, rehypeHighlight]}
      components={{code: Code}}
    >
      {post.content}
    </Markdown>
  )
}

Also, if possible, you should provide props with a stable identity to reduce rerenders even further.

function Code({node, className, children, ...props}) {
  return (
    <div className={className}>
      {children}
    </div>
  )
}

const components = {
  code: Code
}

const rehypePlugins = [
  rehypeRaw,
  rehypeHighlight
]

export function Post({ post }) {
  return (
    <Markdown
      rehypePlugins={rehypePlugins}
      components={components}
    >
      {post.content}
    </Markdown>
  )
}
aeharding commented 6 months ago

This memory leak appears to happen by simply including rehype-remark, even without any actual highlighted content.

If possible, you should provide components with a stable identity to avoid rerenders.

This does not noticeably address the leak in my testing

As others suggest, my current workaround is to use v6.

tgram-3D commented 6 months ago

I'm not sure if anyone else has run into this, but one thing of note is if you downgrade to rehype-highlight 6.0.0 you get a TS error similar to this:

https://github.com/hashicorp/next-mdx-remote/issues/86

The code works fine with no memory leak, and you can add @ts-expect-error to get rid of it as per the above thread, but I'm curious if there's a better solution.

immccn123 commented 3 months ago

This issue seems to be caused by the DOMContentLoaded event listener on the window not being removed in a timely manner. You can type anything on https://remarkjs.github.io/react-markdown/ and use getEventListeners(window).DOMContentLoaded in a Chromium-based browser to compare the length before and after.

https://github.com/user-attachments/assets/eae3e07b-3852-49d2-84eb-7c434ff55c25

window.addEventListener("DOMContentLoaded",...); appears only once in the minimized code.

Results before and after executing the following code:

getEventListeners(window).DOMContentLoaded.forEach(x => window.removeEventListener("DOMContentLoaded", x.listener))

image

Snapshot 1: Before typing
Snapshot 2: After typing
Snapshot 3: After executing the code

wooorm commented 3 months ago

Thanks for investigating this and looking for a proper fix, Imken!

wooorm commented 2 months ago

Thanks again, closed in https://github.com/highlightjs/highlight.js/pull/4095!

github-actions[bot] commented 2 months ago

Hi! This was closed. Team: If this was fixed, please add phase/solved. Otherwise, please add one of the no/* labels.