search

LinkedDataFragments / Client.js

[DEPRECATED] A JavaScript client for Triple Pattern Fragments interfaces.
http://linkeddatafragments.org/
Other
92 stars 35 forks source link

Client crashes with the JavaScript heap out of memory error #28

Closed migalkin closed 7 years ago

migalkin commented 7 years ago

I'm running a C4 query from the set of Complex queries against Fedbench. The query is:

SELECT DISTINCT ?d ?drug5 ?cpd ?enzyme ?equation
WHERE {
?drug1 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/possibleDiseaseTarget> <http://www4.wiwiss.fu-berlin.de/diseasome/resource/diseases/261> .
?drug1 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/target> ?o.
?o <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/genbankIdGene> ?g.
?o <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/locus> ?l.
?o <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/molecularWeight> ?mw.
?o <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/hprdId> ?hp.
?o <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/swissprotName> ?sn.
?o <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/proteinSequence> ?ps.
?o <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/generalReference> ?gr.
?drug <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/target>?o.
?drug <http://www.w3.org/2002/07/owl#sameAs> ?drug5 .
?drug <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/possibleDiseaseTarget> ?d1 . 
?drug <http://www.w3.org/2002/07/owl#sameAs> ?drug5 .
?drug5 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://dbpedia.org/ontology/Drug> .
?drug <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/keggCompoundId> ?cpd .
?enzyme <http://bio2rdf.org/ns/kegg#xSubstrate> ?cpd .
?enzyme <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://bio2rdf.org/ns/kegg#Enzyme> .
?reaction <http://bio2rdf.org/ns/kegg#xEnzyme> ?enzyme .
?reaction <http://bio2rdf.org/ns/kegg#equation> ?equation .
}

All the LDF clients which received the rewritten queries out of this crashed with the following errors. DBpedia:

[Fri Dec 09 2016 11:52:48 GMT+0000 (UTC)] DEBUG TriplePatternIterator 1730 6 {"?drug5":"http://dbpedia.org/resource/Ethynodiol_diacetate"} ?drug type Drug. 4348
[Fri Dec 09 2016 11:52:52 GMT+0000 (UTC)] DEBUG TriplePatternIterator 1731 6 {"?drug5":"http://dbpedia.org/resource/Eticlopride"} ?drug type Drug. 4348
[Fri Dec 09 2016 11:52:59 GMT+0000 (UTC)] DEBUG TriplePatternIterator 1732 6 {"?drug5":"http://dbpedia.org/resource/Eticyclidine"} ?drug type Drug. 4348
[Fri Dec 09 2016 11:53:04 GMT+0000 (UTC)] DEBUG TriplePatternIterator 1733 6 {"?drug5":"http://dbpedia.org/resource/Etidocaine"} ?drug type Drug. 4348
[Fri Dec 09 2016 11:53:08 GMT+0000 (UTC)] DEBUG TriplePatternIterator 1734 6 {"?drug5":"http://dbpedia.org/resource/Etidronic_acid"} ?drug type Drug. 4348
[Fri Dec 09 2016 11:53:15 GMT+0000 (UTC)] DEBUG TriplePatternIterator 1735 6 {"?drug5":"http://dbpedia.org/resource/Etifoxine"} ?drug type Drug. 4348

<--- Last few GCs --->

[1:0x251e9a0]  5585291 ms: Mark-sweep 1399.4 (1451.7) -> 1399.4 (1451.7) MB, 3705.4 / 0.0 ms  allocation failure GC in old space requested
[1:0x251e9a0]  5588710 ms: Mark-sweep 1399.4 (1451.7) -> 1399.3 (1420.7) MB, 3418.6 / 0.0 ms  last resort gc 
[1:0x251e9a0]  5591534 ms: Mark-sweep 1399.3 (1420.7) -> 1399.3 (1420.7) MB, 2824.0 / 0.0 ms  last resort gc 

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x13e30c4c0d11 <JS Object>
    2: _filter [/ldf_rest/node_modules/ldf-client/lib/sparql/DistinctIterator.js:20] [pc=0x36fdb0c9ef5e](this=0x1329cc8b46d9 <a DistinctIterator with map 0x317d31987b79>,item=0x2568eac63b49 <an Object with map 0x317d3196ec09>)
    3: readAndTransformSimple [/ldf_rest/node_modules/asynciterator/asynciterator.js:~1094] [pc=0x36fdb0a4b919](this=0x3f3173ea75d1 <JS Global Object>,self=0x1329cc8b46d9...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0x126450c [node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewInternalizedStringImpl(v8::internal::Handle<v8::internal::String>, int, unsigned int) [node]
 6: v8::internal::StringTable::LookupString(v8::internal::Isolate*, v8::internal::Handle<v8::internal::String>) [node]
 7: v8::internal::Runtime_HasProperty(int, v8::internal::Object**, v8::internal::Isolate*) [node]
 8: 0x36fdb09063a7

Drugbank:

[Fri Dec 09 2016 12:03:14 GMT+0000 (UTC)] INFO HttpClient Requesting http://drugbank-ldh:3000/drugbank?subject=http%3A%2F%2Fwww4.wiwiss.fu-berlin.de%2Fdrugbank%2Fresource%2Fdrug_interactions%2FDB01032_DB00447&predicate=http%3A%2F%2Fwww4.wiwiss.fu-berlin.de%2Fdrugbank%2Fresource%2Fdrugbank%2FinteractionDrug2
[Fri Dec 09 2016 12:03:14 GMT+0000 (UTC)] INFO HttpClient Requesting http://drugbank-ldh:3000/drugbank?subject=http%3A%2F%2Fwww4.wiwiss.fu-berlin.de%2Fdrugbank%2Fresource%2Fdrug_interactions%2FDB00280_DB00187&predicate=http%3A%2F%2Fwww4.wiwiss.fu-berlin.de%2Fdrugbank%2Fresource%2Fdrugbank%2FinteractionDrug2
[Fri Dec 09 2016 12:03:14 GMT+0000 (UTC)] INFO HttpClient Requesting http://drugbank-ldh:3000/drugbank
http://drugbank-ldh:3000/drugbank
SELECT DISTINCT ?drug1 ?mw ?sn ?hp ?o ?l ?ps ?g ?gr WHERE {        ?drug1 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/possibleDiseaseTarget> <http://www4.wiwiss.fu-berlin.de/diseasome/resource/diseases/261> .         ?drug1 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/target> ?o .         ?o <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/genbankIdGene> ?g .         ?o <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/locus> ?l .         ?o <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/molecularWeight> ?mw .         ?o <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/hprdId> ?hp .         ?o <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/swissprotName> ?sn .         ?o <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/proteinSequence> ?ps .         ?o <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/generalReference> ?gr} LIMIT 100000 OFFSET 0

<--- Last few GCs --->

[1:0x22729a0]  6174698 ms: Mark-sweep 1414.2 (1810.0) -> 1413.9 (1809.0) MB, 4729.2 / 0.0 ms  (+ 3.9 ms in 1 steps since start of marking, biggest step 3.9 ms) allocation failure GC in old space requested
[1:0x22729a0]  6181413 ms: Mark-sweep 1413.9 (1809.0) -> 1413.9 (1802.0) MB, 6710.9 / 0.0 ms  (+ 3.7 ms in 1 steps since start of marking, biggest step 3.7 ms) last resort gc 
[1:0x22729a0]  6187466 ms: Mark-sweep 1413.9 (1802.0) -> 1413.8 (1802.0) MB, 6053.4 / 0.0 ms  last resort gc 

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x500963c0d11 <JS Object>
    2: parse [/ldf_rest/node_modules/sparqljs/lib/SparqlParser.js:657] [pc=0x103f81b553cf](this=0x37ecf09df59 <a Parser with map 0x3d96c3e42059>,input=0x139eb186c731 <String[967]: SELECT DISTINCT ?drug1 ?mw ?sn ?hp ?o ?l ?ps ?g ?gr WHERE {        ?drug1 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/possibleDiseaseTarget> <http://www4.wiwiss.fu-berlin.de/diseasome/resource/diseases/...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0x126450c [node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewCode(v8::internal::CodeDesc const&, unsigned int, v8::internal::Handle<v8::internal::Object>, bool, bool, int, bool) [node]
 6: v8::internal::CodeGenerator::MakeCodeEpilogue(v8::internal::MacroAssembler*, v8::internal::EhFrameWriter*, v8::internal::CompilationInfo*, v8::internal::Handle<v8::internal::Object>) [node]
 7: v8::internal::LChunk::Codegen() [node]
 8: v8::internal::HCompilationJob::FinalizeJobImpl() [node]
 9: v8::internal::CompilationJob::FinalizeJob() [node]
10: v8::internal::Compiler::FinalizeCompilationJob(v8::internal::CompilationJob*) [node]
11: v8::internal::OptimizingCompileDispatcher::InstallOptimizedFunctions() [node]
12: v8::internal::Runtime_TryInstallOptimizedCode(int, v8::internal::Object**, v8::internal::Isolate*) [node]
13: 0x103f73c063a7

In other Complex queries two more clients crashed. Linkedmdb:

[Fri Dec 09 2016 12:09:39 GMT+0000 (UTC)] DEBUG TriplePatternIterator 44664 9 {"?drug":"http://data.linkedmdb.org/resource/film/29045","?drug5":"http://mpii.de/yago/resource/Gol_Maal"} ?drug sameAs ?drug5. 1
[Fri Dec 09 2016 12:09:39 GMT+0000 (UTC)] INFO HttpClient Requesting http://linkedmdb-ldh:3000/linkedmdb?subject=http%3A%2F%2Fdata.linkedmdb.org%2Fresource%2Ffilm%2F29048&predicate=http%3A%2F%2Fwww.w3.org%2F2002%2F07%2Fowl%23sameAs&object=http%3A%2F%2Fmpii.de%2Fyago%2Fresource%2FGone_Are_the_Dayes
[Fri Dec 09 2016 12:09:39 GMT+0000 (UTC)] DEBUG TriplePatternIterator 44665 11 {"?drug":"http://data.linkedmdb.org/resource/actor/10808","?drug5":"http://dbpedia.org/resource/Martina_Stella"} ?drug sameAs ?drug5. 1
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

<--- Last few GCs --->

[1:0x340b9a0]  6533835 ms: Mark-sweep 1402.0 (1513.0) -> 1402.0 (1513.0) MB, 2302.7 / 0.0 ms  allocation failure GC in old space requested
[1:0x340b9a0]  6536162 ms: Mark-sweep 1402.0 (1513.0) -> 1402.0 (1483.0) MB, 2326.2 / 0.0 ms  last resort gc 
[1:0x340b9a0]  6538507 ms: Mark-sweep 1402.0 (1483.0) -> 1401.9 (1483.0) MB, 2344.8 / 0.0 ms  last resort gc 

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x15993d1c0d11 <JS Object>
    1: createWriteReq(aka createWriteReq) [net.js:~737] [pc=0x974a6fcf9f2](this=0x15993d104311 <undefined>,req=0xa70a20fae09 <a WriteWrap with map 0x30e08bbff681>,handle=0x327e731f2de9 <a TCP with map 0x24a949067ca9>,data=0x28ed2abae4d1 <String[511]\: GET /linkedmdb?subject=http%3A%2F%2Fdata.linkedmdb.org%2Fresource%2Ffilm%2F29048&predicate=http%3A%2F%2Fwww.w3.org%2F2002%2F07%2Fowl%23sameAs&obj...

 1: node::Abort() [node]
 2: 0x126450c [node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewRawOneByteString(int, v8::internal::PretenureFlag) [node]
 6: v8::internal::String::SlowFlatten(v8::internal::Handle<v8::internal::ConsString>, v8::internal::PretenureFlag) [node]
 7: v8::String::WriteOneByte(unsigned char*, int, int, int) const [node]
 8: node::StringBytes::Write(v8::Isolate*, char*, unsigned long, v8::Local<v8::Value>, node::encoding, int*) [node]
 9: int node::StreamBase::WriteString<(node::encoding)4>(v8::FunctionCallbackInfo<v8::Value> const&) [node]
10: void node::StreamBase::JSMethod<node::StreamWrap, &(int node::StreamBase::WriteString<(node::encoding)4>(v8::FunctionCallbackInfo<v8::Value> const&))>(v8::FunctionCallbackInfo<v8::Value> const&) [node]
11: 0x974a6fcd7af

NYTimes:

[Fri Dec 09 2016 12:15:42 GMT+0000 (UTC)] DEBUG TriplePatternIterator 77326 12 {"?drug":"http://data.nytimes.com/33181370976510004701","?drug5":"http://rdf.freebase.com/ns/en.new_orleans"} ?drug sameAs ?drug5. 1
[Fri Dec 09 2016 12:15:43 GMT+0000 (UTC)] INFO HttpClient Requesting http://nytimes-ldh:3000/nytimes?subject=http%3A%2F%2Fdata.nytimes.com%2F78481746935780803322&predicate=http%3A%2F%2Fwww.w3.org%2F2002%2F07%2Fowl%23sameAs&object=http%3A%2F%2Fdata.nytimes.com%2Ffbr_capital_markets_corporation_org
[Fri Dec 09 2016 12:15:43 GMT+0000 (UTC)] INFO HttpClient Requesting http://nytimes-ldh:3000/nytimes?subject=http%3A%2F%2Fsws.geonames.org%2F5520910%2F&predicate=http%3A%2F%2Fwww.w3.org%2F2002%2F07%2Fowl%23sameAs&object=http%3A%2F%2Fdbpedia.org%2Fresource%2FEctor_County%252C_Texas
[Fri Dec 09 2016 12:15:43 GMT+0000 (UTC)] INFO HttpClient Requesting http://nytimes-ldh:3000/nytimes?subject=http%3A%2F%2Fdata.nytimes.com%2F78213117051040474213&predicate=http%3A%2F%2Fwww.w3.org%2F2002%2F07%2Fowl%23sameAs&object=http%3A%2F%2Fdata.nytimes.com%2Fschrader_paul_per
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

<--- Last few GCs --->

[1:0x29029a0]  6893441 ms: Mark-sweep 1401.9 (1522.0) -> 1401.9 (1522.0) MB, 2233.0 / 0.0 ms  allocation failure GC in old space requested
[1:0x29029a0]  6895780 ms: Mark-sweep 1401.9 (1522.0) -> 1401.9 (1491.0) MB, 2338.1 / 0.0 ms  last resort gc 
[1:0x29029a0]  6898136 ms: Mark-sweep 1401.9 (1491.0) -> 1401.8 (1491.0) MB, 2356.1 / 0.0 ms  last resort gc 

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x1116227c0d11 <JS Object>
    1: /* anonymous */ [/ldf_rest/node_modules/ldf-client/lib/extractors/ControlsExtractor.js:63] [pc=0x1e6f5b3e4bfd](this=0x390c0e1a75d1 <JS Global Object>,property=0x111622755219 <String[5]: first>)
    2: arguments adaptor frame: 3->1
    3: InnerArrayForEach(aka InnerArrayForEach) [native array.js:~777] [pc=0x1e6f5b4e6f40](this=0x111622704311 <undefined>,bk=0x34414bc6c5a9 <JS Function (Shar...

 1: node::Abort() [node]
 2: 0x126450c [node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewTransitionArray(int) [node]
 6: v8::internal::TransitionArray::Insert(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Map>, v8::internal::SimpleTransitionFlag) [node]
 7: v8::internal::Map::CopyReplaceDescriptors(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::DescriptorArray>, v8::internal::Handle<v8::internal::LayoutDescriptor>, v8::internal::TransitionFlag, v8::internal::MaybeHandle<v8::internal::Name>, char const*, v8::internal::SimpleTransitionFlag) [node]
 8: v8::internal::Map::CopyAddDescriptor(v8::internal::Handle<v8::internal::Map>, v8::internal::Descriptor*, v8::internal::TransitionFlag) [node]
 9: v8::internal::Map::CopyWithField(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::FieldType>, v8::internal::PropertyAttributes, v8::internal::Representation, v8::internal::TransitionFlag) [node]
10: v8::internal::Map::TransitionToDataProperty(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::StoreFromKeyed) [node]
11: v8::internal::LookupIterator::PrepareTransitionToDataProperty(v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::StoreFromKeyed) [node]
12: v8::internal::Object::AddDataProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::ShouldThrow, v8::internal::Object::StoreFromKeyed) [node]
13: v8::internal::JSObject::DefineOwnPropertyIgnoreAttributes(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::ShouldThrow, v8::internal::JSObject::AccessorInfoHandling) [node]
14: v8::internal::JSObject::SetOwnPropertyIgnoreAttributes(v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes) [node]
15: 0x10afa60 [node]
16: v8::internal::Runtime_CreateObjectLiteral(int, v8::internal::Object**, v8::internal::Isolate*) [node]
17: 0x1e6f5b0063a7

I'd never expect it would crash with OOM error.

RubenVerborgh commented 7 years ago

I expect that this happens with queries that result in a lot of intermediary results.

Could you try node --max_old_space_size=4096 -- bin/ldf-client …?

Possible duplicate of #24.

RubenVerborgh commented 7 years ago

One additional question:

All the LDF clients which received the rewritten queries

I assume you have used a federated query planner to split up the SPARQL query. Just as a side note, you know that the client also supports federation out of the box? (But maybe you were comparing different federation algorithms of course.)

migalkin commented 7 years ago

@RubenVerborgh yes, we know that the client supports federation out of the box. We are testing a new federation technique and compare it to (including) LDF. How to specify this parameter for Docker container runtime?

CMD ["node", "bin/www"]
RubenVerborgh commented 7 years ago

Great, curious to see those results :+1:

My colleague @rubensworks can help you with Docker.

rubensworks commented 7 years ago

@migalkin I'm not sure what you mean exactly with that Docker command parameter, could you elaborate? Perhaps a different issue should be created if this is not related to this client crash.

The readme contains two examples on how to evaluate queries with the Docker container.

RubenVerborgh commented 7 years ago

@rubensworks The question is how we can start the LDF client with node --max_old_space_size=4096 -- bin/ldf-client … in Docker.

rubensworks commented 7 years ago

You can simply override the Docker entrypoint when running, where you can append any of the regular options afterwards. For example: docker run -it --rm --entrypoint="node --max_old_space_size=4096 -- bin/ldf-client" ldf-client ...

migalkin commented 7 years ago

@rubensworks Thank you for the suggestion, the following config worked for Docker: CMD ["node", "bin/www", "max_old_space_size=4096"]