search

akhoury / nodebb-plugin-import

migrate your old crappy forum to NodeBB
MIT License
78 stars 31 forks source link

JS heap out of memory issue #237

Open reshadi opened 4 years ago

reshadi commented 4 years ago

while importing a large vbullletin forum, during private message import of 1.3M records, the process crashed with the following error. It seems a very huge JS Map is maintained in memory that does run out of JS heap memory.

nodebb_1 | FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory nodebb_1 | nodebb_1 | <--- Last few GCs ---> nodebb_1 | nodebb_1 | [127:0x416d080] 7139062 ms: Mark-sweep 1374.1 (1459.7) -> 1374.1 (1459.7) MB, 418.4 / 0.0 ms allocation failure GC in old space requested nodebb_1 | [127:0x416d080] 7139615 ms: Mark-sweep 1374.1 (1459.7) -> 1374.1 (1427.2) MB, 502.4 / 0.0 ms last resort GC in old space requested nodebb_1 | [127:0x416d080] 7140123 ms: Mark-sweep 1374.1 (1427.2) -> 1374.1 (1426.7) MB, 507.3 / 0.0 ms last resort GC in old space requested nodebb_1 | nodebb_1 | nodebb_1 | <--- JS stacktrace ---> nodebb_1 | nodebb_1 | ==== JS stack trace ========================================= nodebb_1 | nodebb_1 | Security context: 0x264858f258b9 nodebb_1 | 1: deserializeObject(aka deserializeObject) [/usr/src/app/node_modules/bson/lib/bson/parser/deserializer.js:~41] [pc=0x16448d868f1e](this=0x35da052822d1 ,buffer=0xeda1ce5dde9 ,index=0,options=0xeda1ce5ddb9 ,isArray=0x35da052822d1 ) nodebb_1 | 2: deserialize [/usr/src/app/node_modules/bson/lib/bson/bson.js:~133] [pc=0x... nodebb_1 | nodebb_1 | 1: node::Abort() [/usr/local/bin/node] nodebb_1 | 2: 0x8ccf9c [/usr/local/bin/node] nodebb_1 | 3: v8::Utils::ReportOOMFailure(char const, bool) [/usr/local/bin/node] nodebb_1 | 4: v8::internal::V8::FatalProcessOutOfMemory(char const, bool) [/usr/local/bin/node] nodebb_1 | 5: v8::internal::Factory::NewTransitionArray(int) [/usr/local/bin/node] nodebb_1 | 6: v8::internal::TransitionsAccessor::Insert(v8::internal::Handle, v8::internal::Handle, v8::internal::SimpleTransitionFlag) [/usr/local/bin/node] nodebb_1 | 7: v8::internal::Map::ConnectTransition(v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, v8::internal::SimpleTransitionFlag) [/usr/local/bin/node] nodebb_1 | 8: v8::internal::Map::CopyReplaceDescriptors(v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, v8::internal::TransitionFlag, v8::internal::MaybeHandle, char const, v8::internal::SimpleTransitionFlag) [/usr/local/bin/node] nodebb_1 | 9: v8::internal::Map::CopyAddDescriptor(v8::internal::Handle, v8::internal::Descriptor, v8::internal::TransitionFlag) [/usr/local/bin/node] nodebb_1 | 10: v8::internal::Map::CopyWithField(v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, v8::internal::PropertyAttributes, v8::internal::PropertyConstness, v8::internal::Representation, v8::internal::TransitionFlag) [/usr/local/bin/node] nodebb_1 | 11: v8::internal::Map::TransitionToDataProperty(v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, v8::internal::PropertyAttributes, v8::internal::PropertyConstness, v8::internal::Object::StoreFromKeyed, bool) [/usr/local/bin/node] nodebb_1 | 12: v8::internal::LookupIterator::PrepareTransitionToDataProperty(v8::internal::Handle, v8::internal::Handle, v8::internal::PropertyAttributes, v8::internal::Object::StoreFromKeyed) [/usr/local/bin/node] nodebb_1 | 13: v8::internal::StoreIC::LookupForWrite(v8::internal::LookupIterator, v8::internal::Handle, v8::internal::Object::StoreFromKeyed) [/usr/local/bin/node] nodebb_1 | 14: v8::internal::StoreIC::UpdateCaches(v8::internal::LookupIterator*, v8::internal::Handle, v8::internal::Object::StoreFromKeyed, v8::internal::MaybeHandle) [/usr/local/bin/node] nodebb_1 | 15: v8::internal::StoreIC::Store(v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, v8::internal::Object::StoreFromKeyed) [/usr/local/bin/node] nodebb_1 | 16: v8::internal::KeyedStoreIC::Store(v8::internal::Handle, v8::internal::Handle, v8::internal::Handle) [/usr/local/bin/node] nodebb_1 | 17: v8::internal::Runtime_KeyedStoreIC_Miss(int, v8::internal::Object*, v8::internal::Isolate) [/usr/local/bin/node] nodebb_1 | 18: 0x16448d7042fd

  • © Githubissues.
  • Githubissues is a development platform for aggregating issues.