search

datocms / gatsby-source-datocms

Official GatsbyJS source plugin to pull content from DatoCMS
MIT License
140 stars 50 forks source link

JavaScript heap out of memory #123

Closed dennishn closed 3 years ago

dennishn commented 3 years ago

Hello, for some time we have had issues using the gatsby-source-datocms without running into out-of-memory issues.

We have been able to get past this by increasing allowed memory for node - but we are now seeing this happening across more environments (CI envs, OSX and Windows).

Using your gatsby-portfolio example project, using our projects API key and running the gatsby develop task - we get the following out-of-memory error:

success loading DatoCMS schema - 0.778s
success createSchemaCustomization - 0.825s
⠏ source and transform nodes
⠇ loading DatoCMS content

<--- Last few GCs --->

[68807:0x102925000]    61949 ms: Mark-sweep 2018.5 (2050.8) -> 2018.0 (2051.0) MB, 1490.3 / 0.0 ms  (average mu = 0.091, current mu = 0.004) allocation failure scavenge might not succeed
[68807:0x102925000]    63381 ms: Mark-sweep 2018.7 (2051.0) -> 2018.2 (2051.3) MB, 1424.5 / 0.0 ms  (average mu = 0.050, current mu = 0.005) allocation failure scavenge might not succeed

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x166a5cc808d1 <JSObject>
    0: builtin exit frame: defineProperty(this=0x166a5cc80969 <JSFunction Object (sfi = 0x166a906c83e1)>,0x166a9b903571 <Object map = 0x166a192f6b29>,0x166abdf5a581 <String[#10]: nextEvents>,0x166a9b903431 <State map = 0x166af57f8889>,0x166a5cc80969 <JSFunction Object (sfi = 0x166a906c83e1)>)

    1: new constructor(aka State) [0x166a08e6ec91] [/Users/dni/WebstormProjects/gatsby-portfolio/node_m...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x100080c68 node::Abort() [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
 2: 0x100080dec node::errors::TryCatchScope::~TryCatchScope() [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
 3: 0x100185167 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
 4: 0x100185103 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
 5: 0x10030b2f5 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
 6: 0x10030c9c4 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
 7: 0x100309837 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
 8: 0x1003077fd v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
 9: 0x100312fba v8::internal::Heap::AllocateRawWithLightRetry(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
10: 0x100313041 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
11: 0x1002df0d0 v8::internal::Factory::NewFixedArrayWithFiller(v8::internal::RootIndex, int, v8::internal::Object, v8::internal::AllocationType) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
12: 0x1004f79c8 v8::internal::BaseNameDictionary<v8::internal::NameDictionary, v8::internal::NameDictionaryShape>::New(v8::internal::Isolate*, int, v8::internal::AllocationType, v8::internal::MinimumCapacity) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
13: 0x1004c7857 v8::internal::JSObject::MigrateToMap(v8::internal::Isolate*, v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::Map>, int) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
14: 0x1004dfed2 v8::internal::LookupIterator::TransitionToAccessorProperty(v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
15: 0x1004c0b8b v8::internal::JSObject::DefineAccessor(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
16: 0x1004c06fd v8::internal::JSReceiver::ValidateAndApplyPropertyDescriptor(v8::internal::Isolate*, v8::internal::LookupIterator*, bool, v8::internal::PropertyDescriptor*, v8::internal::PropertyDescriptor*, v8::Maybe<v8::internal::ShouldThrow>, v8::internal::Handle<v8::internal::Name>) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
17: 0x1004bfb1d v8::internal::JSReceiver::OrdinaryDefineOwnProperty(v8::internal::LookupIterator*, v8::internal::PropertyDescriptor*, v8::Maybe<v8::internal::ShouldThrow>) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
18: 0x1004bf777 v8::internal::JSReceiver::OrdinaryDefineOwnProperty(v8::internal::Isolate*, v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyDescriptor*, v8::Maybe<v8::internal::ShouldThrow>) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
19: 0x1004bf203 v8::internal::JSReceiver::DefineProperty(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
20: 0x10022a9b9 v8::internal::Builtin_ObjectDefineProperty(int, unsigned long*, v8::internal::Isolate*) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
21: 0x100950a19 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
error Command failed with signal "SIGABRT".

We've also seen this happen for production builds (gatsby build) - but this happens more irregularly.

The easiest way to reproduce is cloning https://github.com/datocms/gatsby-portfolio and setting the DATO_API_TOKEN - please reach out for this or let me know how I can send it to you.

It does look like out-of-memory issues are happening across Gatsby plugins: https://github.com/gatsbyjs/gatsby/issues/15190 https://github.com/gatsbyjs/gatsby/issues/15256 https://github.com/gatsbyjs/gatsby/issues/15540

But most of those issues seems be resolved by either:

matjack1 commented 3 years ago

hey @dennishn I haven't seen this error building the portfolio and I've done it quite a few times, can you maybe specify which node version are you using? Is maybe that th issue? Or can you help me reproduce somehow?

dennishn commented 3 years ago

Hey @matjack1 - the issue is not the gatsby-portfolio, it's just the easiest way to reproduce it using your gatsby example repository.

It's important to note that this happens with our DatoCMS content - I can share our API token with you via email. Using our DatoCMS content, without even fixing the gatsby gql queries (which obv wont work), will cause the memory issue.

I don't think this is a node issue, i've tried using v10.17.0, v12.16.1 and v14.4.0 and the issue happens on all versions.

stefanoverna commented 3 years ago

Yes @dennishn, please reach to us giving us the name of your project at least :D

https://www.datocms.com/support?topics=technical-support/ive-found-a-bug

Edit: corrected the link :)

matjack1 commented 3 years ago

@dennishn @stefanoverna I can reproduce the same issue, we are going to have a look at this and let you know as soon as possible.

dennishn commented 3 years ago

Thank you very much friends :)

stefanoverna commented 3 years ago

Hey @dennishn, can you please try v2.5.1 and see if it works?

dennishn commented 3 years ago

@stefanoverna Unfortunately no luck - still getting the heap out of memory error.

⠙ source and transform nodes
⠏ loading DatoCMS content

<--- Last few GCs --->

[78654:0x102925000]    49730 ms: Mark-sweep 2029.8 (2084.1) -> 2022.2 (2087.8) MB, 1378.6 / 0.0 ms  (average mu = 0.137, current mu = 0.042) allocation failure GC in old space requested
[78654:0x102925000]    51154 ms: Mark-sweep 2024.4 (2087.8) -> 2023.1 (2061.8) MB, 1223.1 / 0.0 ms  (+ 178.2 ms in 34 steps since start of marking, biggest step 8.8 ms, walltime since start of marking 1424 ms) (average mu = 0.079, current mu = 0.016) allo

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x100950919]
    1: StubFrame [pc: 0x1009b92d8]
Security context: 0x0d79179008d1 <JSObject>
    2: _validate [0xd7968f61a59] [/Users/dni/WebstormProjects/superwe-web/node_modules/@hapi/joi/lib/types/any/index.js:564] [bytecode=0xd7984a73659 offset=204](this=0x0d795df13ef1 <Object map = 0xd795b5d0c49>,0x0d79427c7d59 <String[#15]: DatoCmsTextNode>,0x0d790cf89421 <JSObject>,0x0d7968f62c01 <Object map = 0xd795b5c6389>,0x...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

Writing Node.js report to file: report.20200924.133020.78654.0.001.json
Node.js report completed
 1: 0x100080c68 node::Abort() [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
 2: 0x100080dec node::errors::TryCatchScope::~TryCatchScope() [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
 3: 0x100185167 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
 4: 0x100185103 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
 5: 0x10030b2f5 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
 6: 0x10030c9c4 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
 7: 0x100309837 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
 8: 0x1003077fd v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
 9: 0x100312fba v8::internal::Heap::AllocateRawWithLightRetry(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
10: 0x100313041 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
11: 0x1002e035b v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
12: 0x100618718 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
13: 0x100950919 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
14: 0x1009b92d8 Builtins_CreateEmptyArrayLiteralHandler [/Users/dni/.nvm/versions/node/v12.16.1/bin/node]
[1]    78649 abort      gatsby develop

Will try to find time to dig deeper

stefanoverna commented 3 years ago

On my machine it works :/

pieh commented 3 years ago

This is in fact very likely gatsby core issue. I got my hand on one reproductions and was able to make some quick hanges in gatsby core to prevent this (for validation purposes - those changes need to be done properly and not in hacky way). In any case I opened https://github.com/gatsbyjs/gatsby/issues/28916 in gatsby repo to track this issue (as this is not Datocms specific issue)

stefanoverna commented 3 years ago

awesome, thanks!

pieh commented 3 years ago

Ok, found particular culrpint and workaround (at least until we ship it in stable gatsby). The extra memory usage (and actually cpu perf overhead) comes from http://bluebirdjs.com/docs/api/promise.longstacktraces.html in gatsby develop. If you run BLUEBIRD_LONG_STACK_TRACES=0 gatsby develop you should be able to not hit Out-of-memory fatal crash (or at least not because of the bluebird's long stack traces). I will be opening PR in gatsby project so disabling of those happens automatically, but until then using BLUEBIRD_LONG_STACK_TRACES=0 should unblock you

stefanoverna commented 3 years ago

closing this one to avoid duplicates