Error: ENFILE: file table overflow and Too many open files in system

cyberz commented 1 month ago

Vue - Official extension or vue-tsc version

v2.0.28

VSCode version

1.92.0

Vue version

3.3.11

TypeScript version

5.3.3

System Info

System:
    OS: macOS 14.6
    CPU: (10) arm64 Apple M1 Pro
    Memory: 65.58 MB / 16.00 GB
    Shell: 3.2.57 - /bin/bash
  Binaries:
    Node: 20.12.2 - ~/.nvm/versions/node/v20.12.2/bin/node
    Yarn: 1.22.22 - ~/.nvm/versions/node/v20.12.2/bin/yarn
    npm: 10.5.0 - ~/.nvm/versions/node/v20.12.2/bin/npm
  Browsers:
    Chrome: 127.0.6533.90
    Safari: 17.6

Steps to reproduce

While working on a project once a day I have next errors: "ENFILE: file table overflow ..." and as result "Too many open files in system"

Random apps crash and won't launch

First I checked applications that opened a lot of files: lsof -n +c 0 | cut -f1 -d' ' | uniq -c | sort | tail result:

277 Electron
 282 UserEvent
 343 Code\x20H
 380 corespotl
 405 Dropbox
 423 Google
 589 Code\x20H
 737 Google
33370 Code\x20H
65589 Code\x20H

For display PID instead of application name: lsof -n +c 0 | sed -E 's/^[^ ]+[ ]+([^ ]+).*$/\1/' | uniq -c | sort | tail result:

260 83525
 277 8145
 282 1588
 380 52538
 423 76542
 589 16855
16290 92232
16336 8637
33370 8636
65589 92363

Process info: ps aux | grep 92363 result: 92363 0.0 0.4 1596757920 62368 ?? S 2:44PM 0:58.75 /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) /Users/alex/.vscode/extensions/vue.volar-2.0.28/server.js --node-ipc --clientProcessId=92229

Kill process: sudo kill 92363

Check opened files again: lsof -n +c 0 | cut -f1 -d' ' | uniq -c | sort | tail result:

235 com.docker.backend
 277 Electron
 282 UserEventAgent
 302 Dropbox
 380 corespotlightd
 423 Google\x20Chrome
 473 Code\x20Helper\x20(Plugin)
 589 Code\x20Helper\x20(Plugin)
 610 Google\x20Chrome\x20Helper\x20(Renderer)
 835 Code\x20Helper\x20(Plugin)

What is expected?

-

What is actually happening?

Once a day I have next errors: "ENFILE: file table overflow ..." and and as result "Too many open files in system" Random apps crash and won't launch

Link to minimal reproduction

No response

Any additional comments?

No response

davidmatter commented 1 month ago

Can you share your tsconfig?

cyberz commented 1 month ago

tsconfig.json

{
  "files": [],
  "references": [
    {
      "path": "./tsconfig.node.json"
    },
    {
      "path": "./tsconfig.app.json"
    }
  ]
}

tsconfig.node.json

{
  "extends": "@tsconfig/node18/tsconfig.json",
  "include": [
    "vite.config.*",
    "vitest.config.*",
    "cypress.config.*",
    "nightwatch.conf.*",
    "playwright.config.*"
  ],
  "compilerOptions": {
    "composite": true,
    "noEmit": true,
    "module": "ESNext",
    "moduleResolution": "Bundler",
    "types": ["node"]
  }
}

tsconfig.app.json

{
  "extends": "@vue/tsconfig/tsconfig.dom.json",
  "include": ["src/**/*.ts", "src/**/*.d.ts", "src/**/*.tsx", "src/**/*.vue"],
  "exclude": ["src/**/__tests__/*"],
  "compilerOptions": {
    // "moduleResolution": "node",
    "composite": true,
    "noEmit": true,
    "baseUrl": "./",
    "paths": {
      "@/*": ["./src/*"]
    },
  }
}

project based on: https://github.com/TailAdmin/vue-tailwind-admin-dashboard

gundam78 commented 4 weeks ago

I have the same issue. I found too many '/tmp/vue-named-pipe-2.0.28-inferred-0' in lsof.

goldengecko commented 3 weeks ago

I have the same issue on my system, but hadn't managed to track it down as far as you have. It just seems that it only happens when I'm doing development and have the app running with npm run dev and have done lots of changes so hmr has run lots of times.

Once the problem occurs, I have to restart my machine, so haven't been able to gather any particular data about it apart from that general observation.

The general suggestion seems to be to increase the maxfiles and maxfilesperproc settings, but I'm hesitant to do that without understanding the cause of the problem, and what the settings need to be.

cyberz commented 3 weeks ago

The general suggestion seems to be to increase the maxfiles and maxfilesperproc settings,

I think all you'll get is a little more time between problems occurring because file descriptors keep getting bigger and not smaller. When the maxfiles and maxfilesperproc limits are reached, you will again experience problems with the operating system.

kevinmlong commented 2 weeks ago

I just started experiencing this. I thought it had to do with a Chrome update, but perhaps it was the VS Code and/or Extension Updates.

peroneu commented 1 week ago

I too have started experiencing this a few weeks ago and found out it has something to do with the Vue - Official extension. I've made some tests and here are my findings, maybe it helps the devs to pinpoint the issue.

My setup:

MacOS Monterey 12.4
VSCode: 1.92.2 (Universal) / Commit: fee1edb8d6d72a0ddff41e5f71a671c23ed924b9
Vue - Official 2.1.2

When I navigate through a .vue file, for every arrow key that I press, or selection that I make, a few new file descriptors are opened by two Code Helper (plugin) processes. Also, if I resize the VSCode window while the .vue file is the active file, the number of opened files descriptors grows by a dozen per second.

One of the plugin processes opens files named: /tmp/vue-named-pipe-2.1.2-inferred-1 The other process opens files named like this: ->0xf3c0ac39308920db

I happens only if the Vue - Official extension is enabled. As soon as I disable the extension, all the file descriptors are closed.

How I tested it

I created a blank file named Test.vue and opened it in VSCode. It's the only opened file and it can be an empty file. It just has to be a .vue file
I resized the VSCode window several times to make it open a couple hundred file descriptors, so I could find it in the list of processes
In the terminal, I ran the following command to get the pid of the extension: lsof | awk '{print $1, $2}' | sort | uniq -c | sort -nr | grep Code

The output should be something like this:

605 Code\x20H 44465
603 Code\x20H 44501
88 Code\x20H 44425

I got one of the process numbers for the extension (44465 here) and ran the following command to see an update of the file count every 2 seconds: while true; do lsof -p 44465 | wc -l; sleep 2; done
I started resizing the VSCode window and watched the numbers grow in the terminal

Conclusion

At least in my case, it doesn't have anything to do with running the project or "npm run dev". It is enough to have a .vue file (even if it's empty) in the foreground and select something in it or resize the VSCode window, in order for the file descriptor count to grow. If the number of opened file descriptors gets high enough (10000+), it begins to return errors when saving files and eventually other apps in the system start crashing. If I disable the Vue - Official extension, all the files descriptors are closed and everything gets back to normal.

Temporary solution

For now, I made a habit of disabling and enabling the Vue - Official extension every now and then.

I hope this helps.

davidmatter commented 1 week ago

Thanks @peroneu that helps. We're investigating

johnsoncodehk commented 1 week ago

@peroneu thank you for the information! The leaking problem should be solved in https://github.com/vuejs/language-tools/commit/8bc2416283516324f3ec12b3badf839dafcc2a62, you can use the following version temporarily.

volar-2.1.4-alpha.0.vsix.zip

We will also improve the implementation to reduce the need to create new connections.

peroneu commented 1 week ago

@johnsoncodehk wow, that was fast. Much appreciated!

klongmitre commented 1 week ago

I installed the version linked to by @johnsoncodehk and I just ran into the same issue again.

peroneu commented 1 week ago

Today I have updated the extension to v. 2.1.4, which I assume includes the fix from the file linked above. The update did indeed do something, but it doesn't solve the issue. Resizing the VSCode window doesn't increase the number of file descriptors anymore, but navigating through the file with the arrow keys does. Another thing is that before, there were 2 processes, each with a similar number of files opened. Now, there are 3 processes, and one of them is always approximately the sum of the other too. I have no idea if this means anything... :)

18489 Code\x20H 13887
9266 Code\x20H 9822
9266 Code\x20H 9821

The first number is the output from "lsof" and the second is the process id.

Just doing Select All, then Esc for 10 times opens about 90 new file descriptors. After a couple of hours of work, I ended up with almost 40,000 file descriptors opened.

The solution to disable and enable the extension still works, but it would be great to have it fixed.

@davidmatter @johnsoncodehk If there's anything else you'd like me to test, please let me know. Thanks!

davidmatter commented 1 week ago

@peroneu that's strange, I can't reproduce it anymore since 2.1.4. The number of open files is quite steady on my end. I think @johnsoncodehk wanted to have another look at the pipe code anyways. Are you using any kind of special extensions by any chance? Anything that integrates with the tsserver or the vue ecosystem?

johnsoncodehk commented 1 week ago

I can't reproduce it, but I fixed the possible potential problem at https://github.com/vuejs/language-tools/commit/8064338a73cf622ad112c34f4b3091446beaf2de. Can you try the following version to see if it solves the problem?

volar-2.1.6-alpha.0.vsix.zip

peroneu commented 1 week ago

Thanks a lot! I have built the extension from the commit above and it seems that it does indeed solve the issue for good. After about one hour of work I didn't get any superfluous file descriptors. Great job!

 92 Code\x20H 22482
 87 Code\x20H 22484
 86 Code\x20H 25463
 69 Code\x20H 22483
 61 Code\x20H 22492

johnsoncodehk commented 1 week ago

@peroneu thank you for confirming! The fix has been released for 2.1.6, if you still have problems please let me know.

vuejs / language-tools