High memory usage of browser tabs

[dpaetzel]

Describe the bug I'm not sure whether it's a bug but my emanote browser tabs seem to use a lot of RAM; 1GB and more each (as displayed by Firefox's task manager). This has led to system crashes for me if I have too many emanote tabs open and don't have an eye on my system's overall free memory.

To Reproduce Run the emanote server using emanote run and then open the tab in Firefox.

Expected behavior Not that big a memory footprint.

Desktop (please complete the following information): Firefox 108.0.2.

Additional context My Zettelkasten has quite a lot of math formulae (some of which are really long align environments with 50 and more lines), maybe this has something to do with this? However, the Zettels I have open right now only have 20ish math formulae each of which is rather short (only one to two handful of mathematical symbols) and they use 1GB RAM each nevertheless.

[srid]

Could be something to do with Ema's websocket reload logic (which uses morphdom). See https://ema.srid.ca/topics/hot-reload

Here's the JavaScript code if anyone wants to help with this: https://github.com/srid/ema/blob/master/ema/www/ema-shim.js

EDIT: Probably not; see my next comment below.

[dpaetzel]

Just wanted to add: As a workaround I try to regularly close and reopen Emanote browser tabs in order to not lose my entire user session (there's this annoying systemd 252 behaviour where the entire X server gets reaped if one of the subprocesses uses too much RAM).

I did this just now (i.e. close an Emanote tab) and apparently closing this one tab freed over 8GB of RAM (that tab was open for several hours, though). :speak_no_evil:

Sadly, I personally don't have the time right now to help with this.

[srid]

I debugged this using Chrome inspector's memory profiling, and two things cause the memory leak:

1. highlightjs
2. stork re-registering

If you use a pre-stork Emanote (nix run github:srid/emanote/0828297369dc4753134ff67287d0c303111f5c48) with highlightjs removed from your index.yaml and Markdown frontmatter (you could switch to prism which doesn't leak memory), the memory leak should go away.

Obviously, we should fix both these issues. The stork leak is triggered by this re-registering:

https://github.com/srid/emanote/blob/a041483e055cb577db4bfca43bb4d528bd3626d3/default/templates/components/stork/stork-search-head.tpl#L55-L56

[srid]

@dpaetzel Try #412 (nix run github:srid/emanote/stork-leak) along with removing highlightjs from your configuration, and let me know if that helps.

EDIT: I've merged it; so you can just update Emanote and try it.

[dpaetzel]

Awesome, thanks. I'll try it! :slightly_smiling_face:

[srid]

@dpaetzel Have you been able to confirm the fix? Can we close this issue?

[dpaetzel]

I used github:srid/emanote/stork-leak for almost a week now and did not notice any leaks any more. Thank you for fixing this! :slightly_smiling_face:

[dpaetzel]

PS: Note that Emanote tabs still use ~1G of RAM according to Firefox's task manager which is more than I'd expect.

[srid]

@dpaetzel I do not understand. Your earlier comment says the memory leak is fixed; but your later comment indicates otherwise.

Can you monitor memory usage over time. When you first open the tab, how much memory does it use? Then after a few hours (of edits, etc.), how much does it use then?

[dpaetzel]

I'm sorry for replying ambiguously.

I did not have any system crashes since I started using the stork-leak branch (i.e. the fix seems to improve the situation)—whereas beforehand my system crashed several times a day.

However, I still noticed at least once that one of the Emanote tabs still used ~1GB of RAM. I guess this may indicate that there is another (less severe?) leak (which should probably be different issue?)? But I'd certainly have to investigate this more in-depth. A quick check I performed right now showed that adding a paragraph of a few words (2-5 words) leads to the Emanote tab using 10-20 MB more after it performed the auto-reload stuff (after opening the tab, it used ~260 MB). This seems to stack (i.e. after successively adding 10 short paragraphs and waiting in-between for the auto-reload to happen, it uses around 100 MB more than when freshly opened). Upon reloading the tab, the RAM usage drops back to the original amount of ~260 MB. I'll try to gather more data in the next few days.

[srid]

However, I still noticed at least once that one of the Emanote tabs still used ~1GB of RAM. I guess this may indicate that there is another (less severe?) leak (which should probably be different issue?)? But I'd certainly have to investigate this more in-depth. A quick check I performed right now showed that adding a paragraph of a few words (2-5 words) leads to the Emanote tab using 10-20 MB more after it performed the auto-reload stuff (after opening the tab, it used ~260 MB). This seems to stack (i.e. after successively adding 10 short paragraphs and waiting in-between for the auto-reload to happen, it uses around 100 MB more than when freshly opened). Upon reloading the tab, the RAM usage drops back to the original amount of ~260 MB. I'll try to gather more data in the next few days.

Was this happening when highlightjs was still enabled, or disabled?

[dpaetzel]

My index.yaml looks like this:

page:
  headHtml: |
    <script>
      window.MathJax = {
        startup: {
          ready: () => {
            MathJax.startup.defaultReady();
          }
        }
      };
    </script>
    <script async="" id="MathJax-script" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>

Since highlightjs is not enabled explicitly, I think it fulfills your requirement of highlightjs being disabled? Should I disable the mathjax header as well (almost all the Zettels I'm currently working on contain formulae, though, so this is not that straightforward to test for me during normal operation)?

[dpaetzel]

I just reloaded an Emanote tab that I had open for several hours (somewhere around 10 hours) with the effect of freeing 1.7 GB of memory. Firefox's process manager said that it used 2 GB of memory and upon reloading that dropped down to around 300 MB.

[pca006132]

It seems that after doing a few link navigation and reloads, clicking on a link can become really slow, e.g. up to 1 second to switch the link. Firefox can be slower, but chrome can eventually become slow as well.

It seems that tailwind is trying to rewrite the entire DOM, and causing it to be very slow... but I am not a web dev, not sure if this is the real cause. and the GC is taking quite a lot of time on firefox...