arcanis
commented
4 years ago Did fsevents managed to build properly? That's often the main cause for this kind of error.
Closed nkalinov closed 4 years ago
arcanis
commented
4 years ago Did fsevents managed to build properly? That's often the main cause for this kind of error.
nkalinov
commented
4 years ago Sorry but what do you mean, how should I verify this? I don't have any custom builds on fsevents, this is done by yarn, right?
nkalinov
commented
4 years ago hm, interesting, while shooting in the dark, I tried to minimize the watch tree with:
render(
<Provider store={store}>
-- <App />
++ <h1>Hello</h1>
</Provider>,
document.querySelector('#app')
);and recompilation works, so I guess there is some limitation on.... something ... that I didn't have before. Any ideas?
nkalinov
commented
4 years ago Alright, a little update with some speculations: I feel this has something to do with one of the babel and/or webpack plugins. ~What makes me think so is the fact that the recompilation works (not always, but most of the time) when I am updating "legacy" files (written in ES5).~ Nevermind, it started to fail again, all the time, no matter the file. What I tried so far:
Still without success and this is a blocker to migrate our project :(
nkalinov
commented
4 years ago What might be a clue is that sometimes I also get this error:
(node:8074) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 6828)
(node:8074) UnhandledPromiseRejectionWarning: Error: Can't open file: No file descriptors available
Require stack:
- /Users/nkalinov/dev/simonapp/.yarn/cache/watchpack-npm-1.6.0-2e77885616-1.zip/node_modules/watchpack/lib/watcherManager.js
- /Users/nkalinov/dev/simonapp/.yarn/cache/watchpack-npm-1.6.0-2e77885616-1.zip/node_modules/watchpack/lib/watchpack.js
- /Users/nkalinov/dev/simonapp/.yarn/cache/webpack-npm-4.41.5-ec6f53e9d3-1.zip/node_modules/webpack/lib/node/NodeWatchFileSystem.js
- /Users/nkalinov/dev/simonapp/.yarn/cache/webpack-npm-4.41.5-ec6f53e9d3-1.zip/node_modules/webpack/lib/node/NodeEnvironmentPlugin.js
- /Users/nkalinov/dev/simonapp/.yarn/cache/webpack-npm-4.41.5-ec6f53e9d3-1.zip/node_modules/webpack/lib/webpack.js
- /Users/nkalinov/dev/simonapp/.yarn/$$virtual/webpack-dev-server-virtual-d46dc61805/0/cache/webpack-dev-server-npm-3.10.1-34b49b6a2b-1.zip/node_modules/webpack-dev-server/bin/webpack-dev-server.js
at new ZipFS (/Users/nkalinov/dev/simonapp/.pnp.js:27811:15)
at ZipOpenFS.getZipSync (/Users/nkalinov/dev/simonapp/.pnp.js:32309:52)
at ZipOpenFS.makeCallSync (/Users/nkalinov/dev/simonapp/.pnp.js:32208:17)
at ZipOpenFS.existsSync (/Users/nkalinov/dev/simonapp/.pnp.js:31706:17)
at VirtualFS.existsSync (/Users/nkalinov/dev/simonapp/.pnp.js:26230:24)
at extensions.map.find.candidateFile (/Users/nkalinov/dev/simonapp/.pnp.js:39103:28)
at Array.find (<anonymous>)
at applyNodeExtensionResolution (/Users/nkalinov/dev/simonapp/.pnp.js:39101:10)
at resolveUnqualified (/Users/nkalinov/dev/simonapp/.pnp.js:39455:27)
at resolveRequest (/Users/nkalinov/dev/simonapp/.pnp.js:39487:14)
at Object.resolveRequest.maybeLog [as resolveRequest] (/Users/nkalinov/dev/simonapp/.pnp.js:39549:26)
at Function.module_1.Module._resolveFilename (/Users/nkalinov/dev/simonapp/.pnp.js:38778:34)
at Function.module_1.Module._load (/Users/nkalinov/dev/simonapp/.pnp.js:38663:40)
at Module.require (internal/modules/cjs/loader.js:692:17)
at require (internal/modules/cjs/helpers.js:25:18)
at WatcherManager.getDirectoryWatcher (/Users/nkalinov/dev/simonapp/.yarn/cache/watchpack-npm-1.6.0-2e77885616-1.zip/node_modules/watchpack/lib/watcherManager.js:14:25)I am very sorry to say this, but I should raise the white flag (for now) with the PnP migration.
Webpack recompilation works fine only by using @yarnpkg/plugin-node-modules.
I feel this also has something to do with our codebase since we have a few legacy libs, a lot of images and unneeded styles but I was unable to localize the real issue.
The project is a monorepo, using Yarn workspaces.
Our build setup is literally a fork of react-scripts@3.3.1 with the following additions:
babel-plugin-import['@babel/plugin-proposal-decorators', { legacy: true }]@babel/plugin-transform-modules-commonjs.less styles processing: less-loader and import-glob-loaderI will try again in a few weeks, once we are able to get rid of the legacy stuff or if anyone here has an idea...
arcanis
commented
4 years ago That's very curious - we're using PnP at work with a pretty large Webpack app of multiple thousands of files, but we haven't seen this error at all.
The only thing I can think of would be if the fsevents package failed to build (because then watchpack would fallback on the fs.watch function which is known to be vulnerable to file descriptor exhaustion). But if that was the case you should have a message at build time telling you that fsevents fails to build, and a log file.
If by any chance you managed to make a minimal repro I can take a look (I understand it might be difficult considering it likely happens only on your proprietary codebase), but otherwise I have no idea what might cause this, at least for now π
arcanis
commented
4 years ago @nkalinov can you try to run the following? I've just fixed something that might be a factor in the issue, but I'm not sure:
yarn set version from sources --branch 895
yarn install
// then the commands you typically runThanks @arcanis! Just tested a few times and it is very hard to describe the failure pattern (if any) but I will try to:
yarn start -> recompilations: OKyarn start -> first few (~2) recompilations: ERROR, after that: OKyarn cache clean, yarn install, just like that...Up to you to interpret the results correctly but I think it is slightly better...?
./src/app/App.js
Error: Can't open file: No file descriptors available
Require stack:
- /Users/nkalinov/dev/simonapp/.yarn/cache/loader-runner-npm-2.4.0-c414104c2f-2.zip/node_modules/loader-runner/lib/loadLoader.js
- /Users/nkalinov/dev/simonapp/.yarn/cache/loader-runner-npm-2.4.0-c414104c2f-2.zip/node_modules/loader-runner/lib/LoaderRunner.js
- /Users/nkalinov/dev/simonapp/.yarn/cache/webpack-npm-4.41.5-ec6f53e9d3-2.zip/node_modules/webpack/lib/NormalModule.js
- /Users/nkalinov/dev/simonapp/.yarn/cache/webpack-npm-4.41.5-ec6f53e9d3-2.zip/node_modules/webpack/lib/NormalModuleFactory.js
- /Users/nkalinov/dev/simonapp/.yarn/cache/webpack-npm-4.41.5-ec6f53e9d3-2.zip/node_modules/webpack/lib/Compiler.js
- /Users/nkalinov/dev/simonapp/.yarn/cache/webpack-npm-4.41.5-ec6f53e9d3-2.zip/node_modules/webpack/lib/webpack.js
- /Users/nkalinov/dev/simonapp/.yarn/$$virtual/@simon-react-scripts-virtual-33f015bbc0/1/packages/react-scripts/scripts/start.js
at Array.find (<anonymous>)That's in the console (which is truncated I believe to only 1 error). On the webpage I have something slightly different:
Html Webpack Plugin:
Error: /Users/nkalinov/dev/simonapp/.pnp.js:41407
throw firstError;
^
Error: Can't open file: No file descriptors available
Require stack:
- /Users/nkalinov/dev/simonapp/.yarn/$$virtual/html-webpack-plugin-virtual-29e2ac2252/0/cache/html-webpack-plugin-npm-4.0.0-beta.11-4284bef809-2.zip/no de_modules/html-webpack-plugin/index.js
- /Users/nkalinov/dev/simonapp/.yarn/$$virtual/@simon-react-scripts-virtual-33f015bbc0/1/packages/react-scripts/config/webpack.config.js
- /Users/nkalinov/dev/simonapp/.yarn/$$virtual/@simon-react-scripts-virtual-33f015bbc0/1/packages/react-scripts/scripts/start.js
- .pnp.js:30390 new ZipFS
/Users/nkalinov/dev/simonapp/.pnp.js:30390:15
- .pnp.js:34916 ZipOpenFS.getZipSync
/Users/nkalinov/dev/simonapp/.pnp.js:34916:27
- .pnp.js:34815 ZipOpenFS.makeCallSync
/Users/nkalinov/dev/simonapp/.pnp.js:34815:17
- .pnp.js:34313 ZipOpenFS.existsSync
/Users/nkalinov/dev/simonapp/.pnp.js:34313:17
- .pnp.js:28800 VirtualFS.existsSync
/Users/nkalinov/dev/simonapp/.pnp.js:28800:24
- .pnp.js:41715
/Users/nkalinov/dev/simonapp/.pnp.js:41715:28
- Array.find
- .pnp.js:41713 applyNodeExtensionResolution
/Users/nkalinov/dev/simonapp/.pnp.js:41713:10
- .pnp.js:42067 resolveUnqualified
/Users/nkalinov/dev/simonapp/.pnp.js:42067:27
- .pnp.js:42099 resolveRequest
/Users/nkalinov/dev/simonapp/.pnp.js:42099:14Also, maybe of some importance: Sometimes, after I made a change (one change, like a new line), there are 2 recompilations happening subsequently. π I don't have this behavior in "node_modules" mode.
nkalinov
commented
4 years ago @arcanis I think I have something.
I managed to get to a state where I get this error only at the very first recompilation when babel-loader still hasn't created its cache in node_modules/.cache/babel-loader. After the first failure, the recompilation works just fine.
Probably a relevant fact is that In parallel to Yarn2 migration I also refactored our build config and I often switched on and off various babel plugins as well as modules format (using sourceType: 'unambiguous' and changing the way I import modules with import and/or require). Maybe, I am thinking, if switching a file between different module systems also triggers the caching in the same way as point 1. then it might be the same issue...
So now I am able to always reproduce it by rm -rf node_modules/, yarn start, then changing a line somewhere triggers this error the first time.
./src/app/App.js
Error: [BABEL] /Users/nkalinov/dev/simon/packages/simonmarkets-web/src/app/App.js: Can't open file: No file descriptors available
Require stack:
- /Users/nkalinov/dev/simon/packages/babel-preset-react/index.js
- /Users/nkalinov/dev/simon/packages/simonmarkets-web/babel.config.js
- /Users/nkalinov/dev/simon/.yarn/cache/@babel-core-npm-7.8.4-15350b632c-2.zip/node_modules/@babel/core/lib/config/files/module-types.js
- /Users/nkalinov/dev/simon/.yarn/cache/@babel-core-npm-7.8.4-15350b632c-2.zip/node_modules/@babel/core/lib/config/files/configuration.js
- /Users/nkalinov/dev/simon/.yarn/cache/@babel-core-npm-7.8.4-15350b632c-2.zip/node_modules/@babel/core/lib/config/files/index.js
- /Users/nkalinov/dev/simon/.yarn/cache/@babel-core-npm-7.8.4-15350b632c-2.zip/node_modules/@babel/core/lib/index.js
- /Users/nkalinov/dev/simon/.yarn/$$virtual/babel-loader-virtual-0f27a9a4a6/0/cache/babel-loader-npm-8.0.6-ccc68d8d38-2.zip/node_modules/babel-loader/lib/index.js
- /Users/nkalinov/dev/simon/.yarn/cache/loader-runner-npm-2.4.0-c414104c2f-2.zip/node_modules/loader-runner/lib/loadLoader.js
- /Users/nkalinov/dev/simon/.yarn/cache/loader-runner-npm-2.4.0-c414104c2f-2.zip/node_modules/loader-runner/lib/LoaderRunner.js
- /Users/nkalinov/dev/simon/.yarn/cache/webpack-npm-4.41.5-ec6f53e9d3-2.zip/node_modules/webpack/lib/NormalModule.js
- /Users/nkalinov/dev/simon/.yarn/cache/webpack-npm-4.41.5-ec6f53e9d3-2.zip/node_modules/webpack/lib/NormalModuleFactory.js
- /Users/nkalinov/dev/simon/.yarn/cache/webpack-npm-4.41.5-ec6f53e9d3-2.zip/node_modules/webpack/lib/Compiler.js
- /Users/nkalinov/dev/simon/.yarn/cache/webpack-npm-4.41.5-ec6f53e9d3-2.zip/node_modules/webpack/lib/webpack.js
- /Users/nkalinov/dev/simon/.yarn/$$virtual/@simon-react-scripts-virtual-33f015bbc0/1/packages/react-scripts/scripts/start.js (While processing: "/Users/nkalinov/dev/simon/packages/simonmarkets-web/babel.config.js$0")
at Array.find (<anonymous>)
hydrosquall
commented
4 years ago I'm experiencing this same issue while configuring the Sanddance project inside of a react-cosmos project using yarn 2. Will post again if I can make a minimal public repository, but I'm able to trigger this condition every time on a private project.
Update
node-http-proxy). The moment I close another app, the webpack-dev-server is able to recompile successfully.
bartocc
commented
4 years ago Hit the same error with the following env
OS: OSX
Node version: v12.13.1
Yarn version 2.0.0-rc.29.git.20200227.795ecc8eThe code triggering the Error: Can't open file: No file descriptors available error was iterating over each dependency of my project and doing a
crossFs.readFileSync(pkg.packageLocation + '/package.json'));I've been able to remove it by setting useCache: false when instantiating a ZipOpenFS instance with
const libzip = require('@yarnpkg/libzip').getLibzipSync();
const zipOpenFs = new ZipOpenFS({libzip, useCache: false});
const crossFs = new PosixFS(zipOpenFs);Hopefully, this will help identifying the bug π€
yarnbot
commented
4 years ago Hi! π
This issue looks stale, and doesn't feature the reproducible label - which implies that you didn't provide a working reproduction using Sherlock. As a result, it'll be closed in a few days unless a maintainer explicitly vouches for it or you edit your first post to include a formal reproduction (you can use the playground for that).
Note that we require Sherlock reproductions for long-lived issues (rather than standalone git repositories or similar) because we're a small team. Sherlock gives us the ability to check which bugs are still affecting the master branch at any given point, and decreases the amount of code we need to run on our own machines (thus leading to faster bug resolution faster). It helps us help you! π
If you absolutely cannot reproduce a bug on Sherlock (for example because it's a Windows-only issue), a maintainer will have to manually add the upholded label.
rtsao
commented
4 years ago I'm running into this issue with a larger app with roughly 1000 package dependencies.
Merely building the app works fine, but with watch I get the same Error: Can't open file: No file descriptors available.
After some digging, I think I've narrowed down the problem to watchpack, and specifically this method:
One hacky workaround I've found is:
getDirectoryWatcher(directory, options) {
if (DirectoryWatcher === undefined) {
DirectoryWatcher = require("./DirectoryWatcher");
}
options = options || {};
+ const key = directory.includes(".yarn/$$virtual")
+ ? "yarn_virtual" // Shared watcher for virtual packages
+ : directory + " " + JSON.stringify(options);
- const key = directory + " " + JSON.stringify(options);
const watcher = this.directoryWatchers.get(key);
if (watcher === undefined) {
const newWatcher = new DirectoryWatcher(directory, options);
this.directoryWatchers.set(key, newWatcher);
newWatcher.on("closed", () => {
this.directoryWatchers.delete(key);
});
return newWatcher;
}
return watcher;
}This change prevents the file descriptor error -- I assume the watchpack DirectoryWatcher doesn't play nicely with PnP. I'm not sure if it really even makes sense to have filesystem watchers for virtual packages because they don't exist on disk and are immutable. So I modified the caching mechanism so there's only a single DirectoryWatcher for all the virtual packages, which avoids the file descriptor exhaustion problem.
gfortaine
commented
4 years ago Same concern here. Fixed by upgrading to webpack v5 that removed the fsevents dependency, cc @nkalinov @arcanis
merceyz
commented
4 years ago Seems to be fixed, either by something in newer versions of webpack or https://github.com/yarnpkg/berry/pull/1671 so i'll close this
Describe the bug
We are using webpack-dev-server middleware to develop locally. The initial compilation is fine as well as the production builds (via
webpack-cli -p) but as soon as I change a file the dev-server's recompilation fails with:To Reproduce
The minimal information needed to reproduce your issue (ideally a package.json with a single dep). Note that bugs without minimal reproductions might be closed.
IMPORTANT: We strongly prefer reproductions that use Sherlock. Please check our documentation for more information: https://next.yarnpkg.com/advanced/sherlock
Screenshots
If applicable, add screenshots to help explain your problem.
Environment if relevant (please complete the following information):
Additional context
Let me know if webpack conf would be helpful... Any additional tweaks needed to support recompilation/hmr?
Thanks!