nodejs / node

Node.js JavaScript runtime ✨🐢🚀✨
https://nodejs.org
Other
104.27k stars 28.06k forks source link

FS watch memory leak on Windows #52769

Open amcgoogan opened 2 weeks ago

amcgoogan commented 2 weeks ago

Version

v18.20.2

Platform

Microsoft Windows NT 10.0.22631.0 x64

Subsystem

No response

What steps will reproduce the bug?

The following script which continually creates and closes fs watches can reproduce the leak:

const os =  require('os');
const fs =  require('fs');
const path = require('path');
const DIR = path.join(os.tmpdir(), 'fs-watch-leak');
async function test() {
  if (!fs.existsSync(DIR))
    await fs.promises.mkdir(DIR);
  for (let i=1; i<=1000; ++i)
    await fs.promises.writeFile(
      path.join(DIR, `file-${i}.txt`),
      `${i} test file for watching`
    );
  const files = await fs.promises.readdir(DIR);
  for (let loop = 0; ; ++loop) {
    const watches = [];
    for (const file of files)
      watches.push(fs.watch(path.join(DIR, file)));
    for (const watch of watches)
      watch.close();
    process.stdout.write(JSON.stringify({
      loop,
      files: files.length,
      ...process.memoryUsage()
    }) + "\n");
    await new Promise(resolve => setTimeout(resolve, 1000));
  }
}
test();

How often does it reproduce? Is there a required condition?

always

What is the expected behavior? Why is that the expected behavior?

Closing the fs watch should free all memory associated with it, including native memory.

What do you see instead?

We are seeing RSS continually rise while heap total and used remains consistent: image

Additional information

I've tested this with multiple versions of node (16.20.2 and 20.6.0) and on another version of windows (server 2022), and the leak appears to be present on all.

I've also tested this script on linux and the leak does NOT appear to exist there.

juanarbol commented 2 weeks ago

This type of issue is quite hard to debug; qualify as a memory leak is sometimes a bit complicated. If the OS is not requesting that memory back for something else RSS will remain flat. Would you mind to keep this process running after reading/closing those files and see if the RSS lowers again? FWIW I don't a have a windows machine to recreate this

RedYetiDev commented 1 week ago

Hello @Ajmirisorkar,

You've posted several comments unrelated to the current topic. Could you kindly focus on the matter at hand?

Additionally, could you or a member of the team please remove the off topic comments, as they make it difficult to review the issue at hand.