search

appsmithorg / appsmith

Platform to build admin panels, internal tools, and dashboards. Integrates with 25+ databases and any API.
https://www.appsmith.com
Apache License 2.0
34.68k stars 3.75k forks source link

[Bug]: possible memory leak frontend build process #26031

Closed japrogramer closed 8 months ago

japrogramer commented 1 year ago

Is there an existing issue for this?

Description

I was able to build the frontend before on this laptop but suddenly on the release branch I can no longer build the frontend despite having lots of memory

Sentry Auth Token:
Building for non-airgapped Appsmith instances
Cannot find ESLint plugin (ESLintWebpackPlugin).
Creating an optimized production build...
`isModuleDeclaration` has been deprecated, please migrate to `isImportOrExportDeclaration`
    at isModuleDeclaration (/home/archangel/Code/Work/appsmith/app/client/node_modules/@babel/types/lib/validators/generated/index.js:2794:35)
    at PluginPass.Program (/home/archangel/Code/Work/appsmith/app/client/node_modules/babel-plugin-lodash/lib/index.js:102:44)

<--- Last few GCs --->

[6784:0x57d37f0]    64895 ms: Mark-sweep (reduce) 2044.3 (2091.9) -> 2042.3 (2082.4) MB, 368.9 / 0.0 ms  (+ 497.0 ms in 114 steps since start of marking, biggest step 7.0 ms, walltime since start of marking 920 ms) (average mu = 0.306, current mu = 0.359)[6784:0x57d37f0]    65546 ms: Mark-sweep (reduce) 2043.6 (2082.6) -> 2035.2 (2082.6) MB, 648.4 / 0.0 ms  (average mu = 0.193, current mu = 0.004) allocation failure scavenge might not succeed

<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb090e0 node::Abort() [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
 2: 0xa1b70e  [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
 3: 0xce1a20 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
 4: 0xce1dc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
 5: 0xe99435  [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
 6: 0xe99f16  [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
 7: 0xea843e  [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
 8: 0xea8e80 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
 9: 0xeabdfe v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
10: 0xe6d33a v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
11: 0x11e61e6 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
12: 0x15d9e59  [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
/home/archangel/Code/Work/appsmith/app/client/node_modules/react-scripts/scripts/build.js:19
  throw err;
  ^

RpcIpcMessagePortClosedError: Process 6784 exited [SIGABRT].
    at /home/archangel/Code/Work/appsmith/app/client/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/rpc-ipc/RpcIpcMessagePort.js:19:23
    at Generator.next (<anonymous>)
    at /home/archangel/Code/Work/appsmith/app/client/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/rpc-ipc/RpcIpcMessagePort.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/home/archangel/Code/Work/appsmith/app/client/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/rpc-ipc/RpcIpcMessagePort.js:4:12)
    at ChildProcess.handleExit (/home/archangel/Code/Work/appsmith/app/client/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/rpc-ipc/RpcIpcMessagePort.js:18:42)
    at ChildProcess.emit (node:events:513:28)
    at ChildProcess.emit (node:domain:489:12)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:293:12) {
  code: null,
  signal: 'SIGABRT'
}
➜  client git:(release) ✗ free -h
               total        used        free      shared  buff/cache   available
Mem:            14Gi       874Mi        10Gi       8.0Mi       4.1Gi        14Gi
Swap:          2.0Gi       2.8Mi       2.0Gi

Steps To Reproduce

build the frontend code

Public Sample App

No response

Environment

Production

Issue video log

No response

Version

| * 7a9749e49764921d75b1341c743f374481e94b0b (upstream/release) Update top contributors

japrogramer commented 1 year ago

I did try this export NODE_OPTIONS=--max-old-space-size=12000 but it didn't work. I suspect a memory leak.

japrogramer commented 1 year ago

I tried building on a machine with 32gb of ram and I still run out of memory.

export NODE_OPTIONS=--max-old-space-size=24096

free -h
               total        used        free      shared  buff/cache   available
Mem:            31Gi       312Mi        24Gi       0.0Ki       6.4Gi        30Gi
Swap:             0B          0B          0B

....

<--- Last few GCs --->

[4832:0x583c370]   107404 ms: Scavenge 2027.7 (2080.8) -> 2020.1 (2080.8) MB, 4.0 / 0.0 ms  (average mu = 0.682, current mu = 0.658) allocation failure
[4832:0x583c370]   107438 ms: Scavenge 2027.8 (2080.8) -> 2020.2 (2080.8) MB, 3.7 / 0.0 ms  (average mu = 0.682, current mu = 0.658) allocation failure
[4832:0x583c370]   107458 ms: Scavenge 2027.9 (2080.8) -> 2020.3 (2096.8) MB, 4.5 / 0.0 ms  (average mu = 0.682, current mu = 0.658) allocation failure

<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb090e0 node::Abort() [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
 2: 0xa1b70e  [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
 3: 0xce1a20 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
 4: 0xce1dc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
 5: 0xe99435  [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
 6: 0xea90fd v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
 7: 0xeabdfe v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
 8: 0xe6d33a v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
 9: 0x11e61e6 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
10: 0x15d9e59  [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
/home/ubuntu/appscenter/app/client/node_modules/react-scripts/scripts/build.js:19
  throw err;
  ^

RpcIpcMessagePortClosedError: Process 4832 exited [SIGABRT].
    at /home/ubuntu/appscenter/app/client/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/rpc-ipc/RpcIpcMessagePort.js:19:23
    at Generator.next (<anonymous>)
    at /home/ubuntu/appscenter/app/client/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/rpc-ipc/RpcIpcMessagePort.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/home/ubuntu/appscenter/app/client/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/rpc-ipc/RpcIpcMessagePort.js:4:12)
    at ChildProcess.handleExit (/home/ubuntu/appscenter/app/client/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/rpc-ipc/RpcIpcMessagePort.js:18:42)
    at ChildProcess.emit (node:events:513:28)
    at ChildProcess.emit (node:domain:489:12)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:293:12) {
  code: null,
  signal: 'SIGABRT'
}
japrogramer commented 1 year ago

for some reason running git clean -dfx fixes the issue, it seems that the build process creates cache files that create the memory leak in the following builds.

YanceyYu commented 1 year ago

In package.json, start command to modify craco

craco start => craco --max_old_space_size=4096 start

is124 commented 1 year ago

I faced the same issue, tried both of the above commands and added the following to tsconfig.json:

{
  // ...
  "verbatimModuleSyntax": true,
  "ignoreDeprecations": "5.0"
}

I also had to remove "importsNotUsedAsValues": "error" from tsconfig.json. Now it's working!

Cheizr commented 2 months ago

I'm having this problem too when i do yarn start on the client, the system becomes unresponsive, Release v1.38