search

Tampermonkey / tampermonkey

Tampermonkey is the most popular userscript manager, with over 10 million users. It's available for Chrome, Microsoft Edge, Safari, Opera Next, and Firefox.
GNU General Public License v3.0
4.16k stars 416 forks source link

Optimise RAM consuming in Chrome with XHR #1516

Open AlttiRi opened 2 years ago

AlttiRi commented 2 years ago

TL;DR

Info

I use Chromium 101 with Tampermonkey BETA 4.17.6161.

I have a script that fetches 500 MB Blob to download it. I host the file locally in main OS, so it downloads fast, I do tests in a VM (Virtual Machine) to exclude memory consuming by the server (I use live-server).

Demo

Here is the demo: AlttiRi/gm_fetch/demo.user.js It's big, but the main code is only: demo.user.js#L221-L245

Install it and open this link: https://example.com/gm_fetch-demo?url=http://192.168.1.33:8080/1.webm&only-main-demo=1&no-dl=1

As GM_fetch it uses this script: https://github.com/AlttiRi/gm_fetch/blob/master/index.js It's just the extended version that I posted before here: https://github.com/Tampermonkey/tampermonkey/issues/1278#issuecomment-1004568936

With useStream checkbox it uses responseType: "stream" if it's supported (TM Beta supports it), w/o — responseType: "blob".

Test

ViolentMonkey ("blob")

Lets start with ViolentMonkey BETA v2.13.0.19. It does not support responseType: "stream", so ignore the useStream checkbox.

VM does not use additional memory (RAM) for big file downloading at all. image

However, it writes the data 3 times on hard drive in ...\Chrome\User Data\Default\blob_storage\.... image

1500 MB of a temporal data writing to download a 500 MB file. I assume it uses 500 MB to store Blob in background script, then 500 when it sends the data to the content script, and 500 in web page context.

TamperMonkey ("blob")

Let's check how TM works. Disable the useStream checkbox. It also write x3 data to hard drive (to blob cache storage), however, it consumes a lot of memory additionally: image

TamperMonkey ("stream")

Finally, lets check it with stream approach. Use useStream checkbox.

It also writes to the cache folder for blob data, but only 500 BM. 3 times less! (However, sometimes it does not work so fine, and Blob 5MB chunks are also written to Blob cache folder)

Also there are no peaks on the RAM consuming chart: image

Keep in mind that GC (garbage collector) does not free the memory instantly in this case (as you can see on the screenshot). There is no memory leak, you can download big files multiple times. ~But...~

~(Possible?) Memory leak within the extension~

In both cases (with streaming and w/o) the garbage collect works. ~However, with streaming there is a minor memory leak. It's the problem not with my script, but with the extension.~

~Run downloading multiple times (4-6+ times, with streaming) and close the tab. The memory will not down to the initial value. ...then disable the extension and memory will free:~

image

UPD: See my new comment: https://github.com/Tampermonkey/tampermonkey/issues/1516#issuecomment-1143454703

AlttiRi commented 2 years ago

Additional bug (Firefox)

In Firefox with responseType: "stream" TM works VERY slow since chunks from background script are sent with notable delay. There is a delay between each chuck sending, that results to taking a lot of time for sending the response entirely.

58256 ms to download 200 MB with streaming vs 4587 ms with responseType: "blob".

Take a look at the browser console, how slow it logs onprogress callback in comparison with Chrome.

Yes, In Chrome it (the stream approach) also slow downs downloading, but not so critical. It's can be notable only with the very fast downloading, for example, from a browser cache, or downloading of locally hosted files.

AlttiRi commented 2 years ago

BTW, if you disable swap (page) file in Windows with 3 GB of RAM, TM will not be able to download 500 MB file with responseType: "blob":

image

But it downloads this big file fine with responseType: "stream":

image

AlttiRi commented 2 years ago

I have tested it a bit more, there is no memory leak with streaming.

It's just too "special" work of GC.

With the disabled page file GC works better, more aggressive, since it is limited with only 3 GB of RAM.

Probably, because of the unlimited page file in my first tests it was looking like a memory leak, because in that case GC was too "soft".

derjanb commented 2 years ago

You can also force a garbage collection. Just open Tampermonkey's background page and select the performance tab.

image https://twitter.com/chromedevtools/status/967833574639419392

AlttiRi commented 2 years ago

Firefox's problem can be illustrated well with StreamSaver demo:

ff

While in Chrome it works much faster, I even can download a 1.3 GB file* on a machine with only 3 GB of RAM and no page file:

cr

*locally hosted. So, the extension downloads it too fast (and spends RAM for storing), faster than the extension resents it to a user code.

However, in Firefox this demo does not work from the box with the default settings, it requires to use a custom preset of cookies blocking:

image