cotton install error - Too many open files (os error 24)

[jordanbertasso]

I'm getting a too many open files error when running cotton install.

OS: macOS Monterey - 12.6 (21G115) Arch: Apple M1 npm dependencies:

  "dependencies": {
    "@aws-sdk/client-cognito-identity": "^3.183.0",
    "@aws-sdk/client-s3": "^3.183.0",
    "@aws-sdk/credential-provider-cognito-identity": "^3.183.0",
    "@emotion/react": "^11.10.4",
    "@emotion/styled": "^11.10.4",
    "@mui/icons-material": "^5.10.6",
    "@mui/material": "^5.10.8",
    "@mui/system": "^5.10.8",
    "@mui/x-data-grid": "^5.17.5",
    "@mui/x-date-pickers": "^5.0.3",
    "@reduxjs/toolkit": "^1.8.5",
    "@sentry/nextjs": "^7.14.1",
    "@urql/core": "^3.0.3",
    "@urql/exchange-auth": "^1.0.0",
    "axios": "^0.27.2",
    "cookie": "^0.5.0",
    "graphql": "^16.6.0",
    "moment": "^2.29.4",
    "next": "^12.3.1",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "react-redux": "^8.0.4",
    "react-window": "^1.8.7",
    "urql": "^3.0.3"
  },
  "devDependencies": {
    "@graphql-codegen/cli": "^2.13.2",
    "@graphql-codegen/introspection": "^2.2.1",
    "@graphql-codegen/typed-document-node": "^2.3.3",
    "@graphql-codegen/typescript": "^2.7.3",
    "@graphql-codegen/typescript-operations": "^2.5.3",
    "@types/cookie": "^0.5.1",
    "@types/node": "^18.8.1",
    "@types/react": "^18.0.21",
    "@types/react-dom": "^18.0.6",
    "@types/react-window": "^1.8.5",
    "autoprefixer": "^10.4.12",
    "eslint": "^8.24.0",
    "eslint-config-next": "^12.3.1",
    "eslint-config-prettier": "^8.5.0",
    "eslint-plugin-prettier": "^4.2.1",
    "postcss": "^8.4.17",
    "prettier": "^2.7.1",
    "tailwindcss": "^3.1.8",
    "typescript": "^4.8.4"
  }

---

Error and stack trace:

❯ cotton install
⠼ Installed @aws-sdk/eventstream-serde-universal@3.183.0                                                                                                                           +1607 ~1607Error:
   0: Too many open files (os error 24)

Location:
   src/plan.rs:199

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: cotton::plan::install_package with prefix=[] dep=Dependency { name: "@graphql-codegen/schema-ast", version: Version { major: 2, minor: 5, patch: 1, build: [], pre_release: [] }, dist: Dist { tarball: "https://registry.npmjs.org/@graphql-codegen/schema-ast/-/schema-ast-2.5.1.tgz" }, bins: {} }
      at src/plan.rs:213
   1: cotton::plan::install_dep_tree with prefix=[] dep=DependencyTree { root: Dependency { name: "@graphql-codegen/schema-ast", version: Version { major: 2, minor: 5, patch: 1, build: [], pre_release: [] }, dist: Dist { tarball: "https://registry.npmjs.org/@graphql-codegen/schema-ast/-/schema-ast-2.5.1.tgz" }, bins: {} }, children: {} }
      at src/plan.rs:275

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

[danielhuang]

I believe the open file limit has been reached. Usually, for large projects (especially when using file watchers), the limit needs to be increased. I'm not too familiar with macOS, but this post should provide some help.

[jordanbertasso]

Running sudo launchctl limit maxfiles 65536 200000 seems to have fixed it, but I believe that only lives as long as the shell session it was run in.

My concern is other users would run into this as well. Maybe it's worth adding some information about it into the error message or putting it in a FAQ?

npm and yarn don't have the same issue which is why I raised it here

[danielhuang]

NPM and Yarn both run on Node.js, which uses a separate blocking I/O threadpool where async operations are offloaded. The reason they do not have a "too many open files" error is since both write to the filesystem a single file at a time, which does not cause a lot of files to be open at once.

I'll pin this issue so others can find it more easily.