too many open files on "amplify push"

christianguevara commented 2 years ago

Before opening, please confirm:

[X] I have searched for duplicate or closed issues and discussions.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

JavaScript Framework

React

Amplify APIs

Authentication, REST API, GraphQL API, Storage, PubSub

Amplify Categories

auth, storage, function, api

Environment information

``` # Put output below this line System: OS: macOS 11.4 CPU: (8) x64 Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz Memory: 1.41 GB / 16.00 GB Shell: 5.8 - /bin/zsh Binaries: Node: 14.17.1 - ~/.nvm/versions/node/v14.17.1/bin/node Yarn: 1.22.15 - /usr/local/bin/yarn npm: 7.19.0 - ~/.nvm/versions/node/v14.17.1/bin/npm Watchman: 2021.10.04.00 - /usr/local/bin/watchman Browsers: Brave Browser: 94.1.30.87 Chrome: 94.0.4606.71 Safari: 14.1.1 npmPackages: @abhijithvijayan/eslint-config: ^2.5.6 => 2.5.6 @abhijithvijayan/eslint-config-airbnb: ^1.0.2 => 1.0.2 @aws-amplify/api-graphql: ^2.2.9 => 2.2.9 @aws-amplify/api-rest: ^2.0.20 => 2.0.20 @aws-amplify/auth: ^4.3.10 => 4.3.10 @aws-amplify/core: ^4.3.2 => 4.3.2 @aws-amplify/storage: ^4.4.3 => 4.4.3 @babel/eslint-parser: ^7.15.8 => 7.15.8 @sentry/react: ^6.13.3 => 6.13.3 @sentry/tracing: ^6.13.3 => 6.13.3 @typescript-eslint/eslint-plugin: ^4.33.0 => 4.33.0 @typescript-eslint/parser: ^4.33.0 => 4.33.0 (3.10.1) @u-wave/react-vimeo: ^0.9.5 => 0.9.5 babel-eslint: ^10.1.0 => 10.1.0 baseui: ^10.3.0 => 10.3.0 body-scroll-lock: ^3.1.5 => 3.1.5 braintree-web: ^3.82.0 => 3.82.0 d3: ^7.1.1 => 7.1.1 date-fns: ^2.25.0 => 2.25.0 (1.30.1) date-fns-timezone: ^0.1.4 => 0.1.4 date-fns-tz: ^1.1.6 => 1.1.6 eslint: ^7.32.0 => 7.32.0 eslint-config-prettier: ^7.2.0 => 7.2.0 eslint-plugin-import: ^2.24.2 => 2.24.2 eslint-plugin-jsx-a11y: ^6.4.1 => 6.4.1 eslint-plugin-node: ^11.1.0 => 11.1.0 eslint-plugin-prettier: ^4.0.0 => 4.0.0 eslint-plugin-react: ^7.26.1 => 7.26.1 eslint-plugin-react-hooks: ^4.2.0 => 4.2.0 fast-deep-equal: ^3.1.3 => 3.1.3 find-and: ^1.3.1 => 1.3.1 focus-trap-react: ^8.8.1 => 8.8.1 framer-motion: ^4.1.17 => 4.1.17 graphql-retain-transformer: ^1.0.0 => 1.0.0 hashids: ^2.2.8 => 2.2.8 hashids-esm: undefined () i18next: ^20.6.1 => 20.6.1 i18next-browser-languagedetector: ^6.1.2 => 6.1.2 i18next-xhr-backend: ^3.2.2 => 3.2.2 ismounted: ^0.1.8 => 0.1.8 marked: ^2.1.3 => 2.1.3 match-sorter: ^6.3.1 => 6.3.1 node-sass: ^6.0.1 => 6.0.1 prettier: ^2.4.1 => 2.4.1 prop-types: ^15.7.2 => 15.7.2 qrcode.react: ^1.0.1 => 1.0.1 query-string: ^7.0.1 => 7.0.1 (4.3.4) querystringify: ^2.2.0 => 2.2.0 randomatic: ^3.1.1 => 3.1.1 react: ^17.0.2 => 17.0.2 (16.14.0) react-add-to-calendar: ^0.1.5 => 0.1.5 react-beforeunload: ^2.5.2 => 2.5.2 react-dom: ^17.0.2 => 17.0.2 (16.14.0) react-dom-confetti: ^0.2.0 => 0.2.0 react-ga: ^3.3.0 => 3.3.0 react-hot-loader: ^4.13.0 => 4.13.0 react-hotkeys-hook: ^3.4.3 => 3.4.3 react-i18next: ^11.12.0 => 11.12.0 react-icons: ^4.3.1 => 4.3.1 react-infinite-scroller: ^1.2.4 => 1.2.4 react-jinke-music-player: ^4.24.2 => 4.24.2 react-mailcheck: ^1.2.0 => 1.2.0 react-markdown: ^6.0.3 => 6.0.3 react-mde: ^11.5.0 => 11.5.0 react-pro-sidebar: ^0.7.1 => 0.7.1 react-router-dom: ^5.3.0 => 5.3.0 react-router-scroll-memory: ^2.0.7 => 2.0.7 react-scripts: ^4.0.3 => 4.0.3 react-snapchat-pixel: ^1.1.1 => 1.1.1 react-sortable-tree: ^2.8.0 => 2.8.0 react-timeago: ^6.2.1 => 6.2.1 react-transition-group: ^4.4.2 => 4.4.2 (2.9.0) react-transition-group/CSSTransition: undefined () react-transition-group/ReplaceTransition: undefined () react-transition-group/SwitchTransition: undefined () react-transition-group/Transition: undefined () react-transition-group/TransitionGroup: undefined () react-transition-group/TransitionGroupContext: undefined () react-transition-group/config: undefined () react-truncate: ^2.4.0 => 2.4.0 react-use-intercom: ^1.4.0 => 1.4.0 recharts: 2.0.0-beta.3 => 2.0.0-beta.3 rehype-raw: ^6.0.0 => 6.0.0 remark-gfm: ^1.0.0 => 1.0.0 slugify: ^1.6.1 => 1.6.1 source-map-explorer: ^2.5.2 => 2.5.2 stopword: ^1.0.11 => 1.0.11 styletron-engine-atomic: ^1.4.8 => 1.4.8 styletron-react: ^6.0.1 => 6.0.1 supports-webp: ^2.0.1 => 2.0.1 timezone-support: ^2.0.2 => 2.0.2 (1.8.1) twilio-client: ^1.14.0 => 1.14.0 use-clipboard-copy: ^0.2.0 => 0.2.0 use-debounce: ^7.0.0 => 7.0.0 use-google-autocomplete: ^0.1.0 => 0.1.0 validator: ^13.6.0 => 13.6.0 npmGlobalPackages: @aws-amplify/cli: 5.6.0 ngrok: 4.0.1 npm: 7.19.0 ```

Describe the bug

After some schema updates/additions, executing amplify push --y generates the following error:

✔ Successfully pulled backend environment dev from the cloud.

    Current Environment: dev
──────────┬────────────────────────┬───────────┬───────────────────┐
│ Category │ Resource name          │ Operation │ Provider plugin   │
├──────────┼────────────────────────┼───────────┼───────────────────┤
│ Api      │ company                │ Update    │ awscloudformation │

GraphQL schema compiled successfully.

Edit your schema at /Users/xxxx/amplify/backend/api/company/schema.graphql or place .graphql files in a directory at /Users/xxxx/amplify/backend/api/company/schema

⠴ Uploading files...2021-10-11T20:21:36.958Z - error: uncaughtException: EMFILE: too many open files, open '/Users/xxxx/amplify/backend/api/company/build/states/initial-stack/states/01/states/initial-stack/states/01/resolvers/Subscription.onCreateTemporaryPhone.res.vtl' date=Mon Oct 11 2021 15:21:36 GMT-0500 (Ecuador Time), pid=6548, uid=501, gid=20, cwd=/Users/xxxx, execPath=/Users/yyyy/.nvm/versions/node/v14.17.1/bin/node, version=v14.17.1, argv=[/Users/yyyy/.nvm/versions/node/v14.17.1/bin/node, /Users/yyyy/.nvm/versions/node/v14.17.1/bin/amplify, push, --y], rss=2396909568, heapTotal=2172190720, heapUsed=2105630496, external=95883634, arrayBuffers=93705767, loadavg=[6.455078125, 6.59912109375, 6.79296875], uptime=3184, trace=[], stack=[Error: EMFILE: too many open files, open '/Users/xxxx/amplify/backend/api/company/build/states/initial-stack/states/01/states/initial-stack/states/01/resolvers/Subscription.onCreateTemporaryPhone.res.vtl']

Then I checked for limits using ulimit -a and got:

-t: cpu time (seconds)              unlimited
-f: file size (blocks)              unlimited
-d: data seg size (kbytes)          unlimited
-s: stack size (kbytes)             8192
-c: core file size (blocks)         0
-v: address space (kbytes)          unlimited
-l: locked-in-memory size (kbytes)  unlimited
-u: processes                       2784
-n: file descriptors                65536

I DO have watchman installed using homebrew. I tried also to increase the limit using ulimit -n 900536 I tried also to upgrade/downgrade amplify CLI version.

Expected behavior

Deploy successful

Reproduction steps

Hard to describe since is a big platform and it can't be reproduced easily but...

Had everything working as normal
Add a new field into a collection: smsOffers: [LeadOffers] @connection(keyName: "ByLead")
Deployment started to fail

Relevant types

type Lead
@model(subscriptions: null)
@key(name: "ByLeadMagnetSite", fields: ["leadMagnetSiteId"], queryField: "leadsByLeadMagnetSite")
@key(name: "ByEmail", fields: ["email"], queryField: "leadsByEmail")
@key(name: "ByPhoneNumber", fields: ["phoneNumber"], queryField: "leadsByPhoneNumber")
@key(name: "ByCountryCode", fields: ["countryCode"], queryField: "leadsByCountryCode")
{
  id: ID!
  hash: String
  smsOffers: [LeadOffers] @connection(keyName: "ByLead")
  webOffers: [LeadOffers]
  mailingOffers: [LeadOffers]
}

type LeadOffers
@model(subscriptions: null)
@key(name: "ByCampaign", fields: ["campaign"], queryField: "leadOffersByCampaign")
@key(name: "ByHash", fields: ["hash"], queryField: "leadOffersByHash")
@key(name: "ByLead", fields: ["leadId"], queryField: "leadsOffersByLead")
{
  id: ID! # unique ID
  campaign: String # For tracking purposes
  friendlyName: String
  hash: String
  expiresAt: AWSDateTime!
  pricePoint: String
  basePrice: Int
  trialPrice: Int
  trialPeriodInDays: Int,
  usedIn: LeadOfferPurpose
  leadId: ID!
  lead: Lead @connection(fields: ["leadId"])
  createdAt: AWSDateTime!
  updatedAt: AWSDateTime
}

enum LeadOfferPurpose {
  EMAIL
  SMS
  WEB
}

Code Snippet

// Put your code below this line.

Log output

``` // Put your logs below this line ```

aws-exports.js

No response

Manual configuration

No response

Additional configuration

No response

Mobile Device

No response

Mobile Operating System

No response

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

No response

christianguevara commented 2 years ago

FYI, to make it work @connection(keyName: "ByLead") I added the connection directive one at a time. It means, add it and then deploy, and so on. Weird.

github-actions[bot] commented 1 year ago

This issue has been automatically locked since there hasn't been any recent activity after it was closed. Please open a new issue for related bugs.

Looking for a help forum? We recommend joining the Amplify Community Discord server *-help channels or Discussions for those types of questions.

aws-amplify / amplify-js