search

appsmithorg / appsmith

Platform to build admin panels, internal tools, and dashboards. Integrates with 25+ databases and any API.
https://www.appsmith.com
Apache License 2.0
34.08k stars 3.68k forks source link

[Bug]: possible memory leak frontend build process #26031

Closed japrogramer closed 7 months ago

japrogramer commented 1 year ago

Is there an existing issue for this?

Description

I was able to build the frontend before on this laptop but suddenly on the release branch I can no longer build the frontend despite having lots of memory

Sentry Auth Token:
Building for non-airgapped Appsmith instances
Cannot find ESLint plugin (ESLintWebpackPlugin).
Creating an optimized production build...
`isModuleDeclaration` has been deprecated, please migrate to `isImportOrExportDeclaration`
    at isModuleDeclaration (/home/archangel/Code/Work/appsmith/app/client/node_modules/@babel/types/lib/validators/generated/index.js:2794:35)
    at PluginPass.Program (/home/archangel/Code/Work/appsmith/app/client/node_modules/babel-plugin-lodash/lib/index.js:102:44)

<--- Last few GCs --->

[6784:0x57d37f0]    64895 ms: Mark-sweep (reduce) 2044.3 (2091.9) -> 2042.3 (2082.4) MB, 368.9 / 0.0 ms  (+ 497.0 ms in 114 steps since start of marking, biggest step 7.0 ms, walltime since start of marking 920 ms) (average mu = 0.306, current mu = 0.359)[6784:0x57d37f0]    65546 ms: Mark-sweep (reduce) 2043.6 (2082.6) -> 2035.2 (2082.6) MB, 648.4 / 0.0 ms  (average mu = 0.193, current mu = 0.004) allocation failure scavenge might not succeed

<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb090e0 node::Abort() [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
 2: 0xa1b70e  [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
 3: 0xce1a20 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
 4: 0xce1dc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
 5: 0xe99435  [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
 6: 0xe99f16  [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
 7: 0xea843e  [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
 8: 0xea8e80 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
 9: 0xeabdfe v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
10: 0xe6d33a v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
11: 0x11e61e6 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
12: 0x15d9e59  [/home/archangel/.nvm/versions/node/v16.20.1/bin/node]
/home/archangel/Code/Work/appsmith/app/client/node_modules/react-scripts/scripts/build.js:19
  throw err;
  ^

RpcIpcMessagePortClosedError: Process 6784 exited [SIGABRT].
    at /home/archangel/Code/Work/appsmith/app/client/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/rpc-ipc/RpcIpcMessagePort.js:19:23
    at Generator.next (<anonymous>)
    at /home/archangel/Code/Work/appsmith/app/client/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/rpc-ipc/RpcIpcMessagePort.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/home/archangel/Code/Work/appsmith/app/client/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/rpc-ipc/RpcIpcMessagePort.js:4:12)
    at ChildProcess.handleExit (/home/archangel/Code/Work/appsmith/app/client/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/rpc-ipc/RpcIpcMessagePort.js:18:42)
    at ChildProcess.emit (node:events:513:28)
    at ChildProcess.emit (node:domain:489:12)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:293:12) {
  code: null,
  signal: 'SIGABRT'
}
➜  client git:(release) ✗ free -h
               total        used        free      shared  buff/cache   available
Mem:            14Gi       874Mi        10Gi       8.0Mi       4.1Gi        14Gi
Swap:          2.0Gi       2.8Mi       2.0Gi

Steps To Reproduce

build the frontend code

Public Sample App

No response

Environment

Production

Issue video log

No response

Version

| * 7a9749e49764921d75b1341c743f374481e94b0b (upstream/release) Update top contributors

japrogramer commented 1 year ago

I did try this export NODE_OPTIONS=--max-old-space-size=12000 but it didn't work. I suspect a memory leak.

japrogramer commented 1 year ago

I tried building on a machine with 32gb of ram and I still run out of memory.

export NODE_OPTIONS=--max-old-space-size=24096

free -h
               total        used        free      shared  buff/cache   available
Mem:            31Gi       312Mi        24Gi       0.0Ki       6.4Gi        30Gi
Swap:             0B          0B          0B

....

<--- Last few GCs --->

[4832:0x583c370]   107404 ms: Scavenge 2027.7 (2080.8) -> 2020.1 (2080.8) MB, 4.0 / 0.0 ms  (average mu = 0.682, current mu = 0.658) allocation failure
[4832:0x583c370]   107438 ms: Scavenge 2027.8 (2080.8) -> 2020.2 (2080.8) MB, 3.7 / 0.0 ms  (average mu = 0.682, current mu = 0.658) allocation failure
[4832:0x583c370]   107458 ms: Scavenge 2027.9 (2080.8) -> 2020.3 (2096.8) MB, 4.5 / 0.0 ms  (average mu = 0.682, current mu = 0.658) allocation failure

<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb090e0 node::Abort() [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
 2: 0xa1b70e  [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
 3: 0xce1a20 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
 4: 0xce1dc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
 5: 0xe99435  [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
 6: 0xea90fd v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
 7: 0xeabdfe v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
 8: 0xe6d33a v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
 9: 0x11e61e6 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
10: 0x15d9e59  [/home/ubuntu/.nvm/versions/node/v16.20.1/bin/node]
/home/ubuntu/appscenter/app/client/node_modules/react-scripts/scripts/build.js:19
  throw err;
  ^

RpcIpcMessagePortClosedError: Process 4832 exited [SIGABRT].
    at /home/ubuntu/appscenter/app/client/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/rpc-ipc/RpcIpcMessagePort.js:19:23
    at Generator.next (<anonymous>)
    at /home/ubuntu/appscenter/app/client/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/rpc-ipc/RpcIpcMessagePort.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/home/ubuntu/appscenter/app/client/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/rpc-ipc/RpcIpcMessagePort.js:4:12)
    at ChildProcess.handleExit (/home/ubuntu/appscenter/app/client/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/rpc-ipc/RpcIpcMessagePort.js:18:42)
    at ChildProcess.emit (node:events:513:28)
    at ChildProcess.emit (node:domain:489:12)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:293:12) {
  code: null,
  signal: 'SIGABRT'
}
japrogramer commented 1 year ago

for some reason running git clean -dfx fixes the issue, it seems that the build process creates cache files that create the memory leak in the following builds.

YanceyYu commented 12 months ago

In package.json, start command to modify craco

craco start => craco --max_old_space_size=4096 start

is124 commented 11 months ago

I faced the same issue, tried both of the above commands and added the following to tsconfig.json:

{
  // ...
  "verbatimModuleSyntax": true,
  "ignoreDeprecations": "5.0"
}

I also had to remove "importsNotUsedAsValues": "error" from tsconfig.json. Now it's working!

Cheizr commented 1 month ago

I'm having this problem too when i do yarn start on the client, the system becomes unresponsive, Release v1.38