search

jupyter-book / mystmd

Command line tools for working with MyST Markdown.
https://mystmd.org/guide
MIT License
219 stars 65 forks source link

JavaScript heap out of memory #1535

Open spisakt opened 2 months ago

spisakt commented 2 months ago

After a few seconds/minutesmyst start terminates with:

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory

myst version: v1.3.8

Description

Issue faced with a very large mystmd project (including jupyter notebooks with large figures).

Proposed solution

The problem seems to be solved by setting:

export NODE_OPTIONS=--max-old-space-size=8192

Additional notes

Full error message:

<--- Last few GCs --->

[65236:0x150008000] 93128 ms: Mark-Compact 3997.7 (4139.6) -> 3985.1 (4143.1) MB, 856.88 / 0.00 ms (average mu = 0.314, current mu = 0.015) allocation failure; scavenge might not succeed [65236:0x150008000] 94703 ms: Mark-Compact 4001.0 (4143.1) -> 3988.4 (4146.9) MB, 1562.00 / 0.00 ms (average mu = 0.141, current mu = 0.008) allocation failure; scavenge might not succeed

<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory ----- Native stack trace -----

1: 0x1043f8b44 node::OOMErrorHandler(char const, v8::OOMDetails const&) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 2: 0x1045803ec v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate, char const, v8::OOMDetails const&) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 3: 0x104754ac0 v8::internal::Heap::GarbageCollectionReasonToString(v8::internal::GarbageCollectionReason) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 4: 0x10475359c v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 5: 0x104749db4 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 6: 0x10474a614 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 7: 0x10472f684 v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 8: 0x104b17394 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long, v8::internal::Isolate) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 9: 0x104e74c44 Builtins_CEntry_Return1_ArgvOnStack_NoBuiltinExit [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 10: 0x104debff4 Builtins_StringSubstring [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 11: 0x10a3340e0 12: 0x10a9e1820 13: 0x10a3ed980 14: 0x10a59caa0 15: 0x104e23210 Builtins_AsyncFunctionAwaitResolveClosure [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 16: 0x104ed0fb8 Builtins_PromiseFulfillReactionJob [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 17: 0x104e12b94 Builtins_RunMicrotasks [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 18: 0x104dea3f4 Builtins_JSRunMicrotasksEntry [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 19: 0x1046c1f40 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate, v8::internal::(anonymous namespace)::InvokeParams const&) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 20: 0x1046c242c v8::internal::(anonymous namespace)::InvokeWithTryCatch(v8::internal::Isolate, v8::internal::(anonymous namespace)::InvokeParams const&) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 21: 0x1046c2608 v8::internal::Execution::TryRunMicrotasks(v8::internal::Isolate, v8::internal::MicrotaskQueue) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 22: 0x1046e97d4 v8::internal::MicrotaskQueue::RunMicrotasks(v8::internal::Isolate) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 23: 0x1046e9f70 v8::internal::MicrotaskQueue::PerformCheckpoint(v8::Isolate) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 24: 0x104324c4c node::InternalCallbackScope::Close() [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 25: 0x104324fe4 node::InternalMakeCallback(node::Environment, v8::Local, v8::Local, v8::Local, int, v8::Local, node::async_context) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 26: 0x10433b104 node::AsyncWrap::MakeCallback(v8::Local, int, v8::Local) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 27: 0x1043fe320 node::fs::FSReqCallback::Resolve(v8::Local) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 28: 0x1043ffde8 node::fs::AfterScanDir(uv_fs_s) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 29: 0x1043f2b2c node::MakeLibuvRequestCallback<uv_fs_s, void ()(uv_fs_s)>::Wrapper(uv_fs_s) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 30: 0x104dc7ea0 uvwork_done [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 31: 0x104dcb8f0 uv__async_io [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 32: 0x104ddd9c8 uvio_poll [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 33: 0x104dcbeb4 uv_run [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 34: 0x1043256f0 node::SpinEventLoopInternal(node::Environment) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 35: 0x1044387c0 node::NodeMainInstance::Run(node::ExitCode, node::Environment*) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 36: 0x1044384d4 node::NodeMainInstance::Run() [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 37: 0x1043c07ac node::Start(int, char**) [/Users/tspisak/.nvm/versions/node/v20.14.0/bin/node] 38: 0x199f220e0 start [/usr/lib/dyld] zsh: abort myst start