Web Workers use 7x more memory than in Chrome, and terminating them doesn't remove all the memory

[josephrocca]

Version: Tested in Deno 1.46.x and 2.x

The code below creates 100 workers, then waits 5 seconds, then terminates them, and repeats.

• In Chrome the memory usage spikes to +150mb, and then back down to zero each cycle.
• In Deno, it spikes to +1000mb, but doesn't go back down to zero. It accumlates memory per cycle
  • EDIT: up to a maximum of +1500mb, so this extra 500mb is potentially just due to some sort of resource pool, or something - maybe expected behavior
  • EDIT 2: It does continue creeping up slowly - after 100k workers created+destroyed, it's now at 700mb
  • EDIT 3: After 500k workers created+destroyed, it's now at about 2000mb leaked.

    
    if(self.Deno) {
    console.log(Deno.memoryUsage());
    setInterval(() => {
    console.log(Deno.memoryUsage());
    }, 1000);
    }

while(1) {

let workers = []; for(let i = 0; i < 100; i++) { const workerBlob = new Blob([ delete globalThis.Deno; self.onmessage = async function(e) { self.postMessage('Hello from Worker!'); } ], {type:"application/javascript"});

const workerURL = URL.createObjectURL(workerBlob);
const worker = new Worker(workerURL, {type:"module"});

worker.onmessage = function(e) {
  URL.revokeObjectURL(workerURL);
  console.log('Received from worker:', e.data);
};
worker.postMessage('Hello from main thread!');

workers.push(worker);

}

await new Promise(r => setTimeout(r, 5*1000));

for(let w of workers) w.terminate(); workers = [];

}

[Hajime-san]

Maybe this could be related about memory usage.

• https://github.com/denoland/deno/issues/11561#issuecomment-894762539

I think the slowdown is a combo of the byte array copy in op_postmessage and Deno.core.serialize and Deno.core.deserialize being seperate calls.

[nathanwhit]

I believe the cause of the increased memory usage is due to us not being able to enable shared RO heap in V8 (the reason is rather complicated, but once V8 fixes the issue we'll be able to enable it). So where chrome has one copy of globals shared across all isolates, we have a copy per isolate. The globals take up about 7-8MB, which would account for much of the extra memory.

Not sure about the memory not being freed, though

[josephrocca]

@nathanwhit Thanks for your comment! I've investigated this a bit further, and it looks like the "leaked" memory actually maxes out at about 500mb, no matter how many workers are created+destroyed, so this may actually be expected behavior - i.e. due to some sort of reuse resource pool, or something. This is a non-issue for me, so unless you suspect this is buggy behavior, this is probably not worth investigating further for now.

• Edit: Actually, it does continue creeping up slowly - after 100k workers created+destroyed, it reached 700mb, and after 500k workers it reached 2000mb leaked. Much slower leak than originally reported though.

RE the "shared RO heap", I'm looking forward to that. Do you know of any v8 crbug or Deno issue/pull request that I can subscribe to for updates?

[nemosmithasf]

Wanted to add volume to this: I also experience memory leaking from large amounts of workers being created/destroyed