dkpro / dkpro-core

Collection of software components for natural language processing (NLP) based on the Apache UIMA framework.
https://dkpro.github.io/dkpro-core
Other
196 stars 67 forks source link

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space #1354

Closed jhammen closed 5 years ago

jhammen commented 5 years ago

Attempting to parse French using Stanford components:

Running the resulting pipeline results in java.lang.OutOfMemoryError: Java heap space

SimplePipeline.runPipeline(reader, seg, parser, writer);

VM args do not help: -Xmx2g -Xms1g etc. same result

full stack trace:


Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at edu.stanford.nlp.parser.lexparser.ExhaustiveDependencyParser.createArrays(ExhaustiveDependencyParser.java:803)
    at edu.stanford.nlp.parser.lexparser.ExhaustiveDependencyParser.parse(ExhaustiveDependencyParser.java:183)
    at edu.stanford.nlp.parser.lexparser.LexicalizedParserQuery.parseInternal(LexicalizedParserQuery.java:283)
    at edu.stanford.nlp.parser.lexparser.LexicalizedParserQuery.parse(LexicalizedParserQuery.java:637)
    at de.tudarmstadt.ukp.dkpro.core.stanfordnlp.StanfordParser.process(StanfordParser.java:420)
    at org.apache.uima.analysis_component.JCasAnnotator_ImplBase.process(JCasAnnotator_ImplBase.java:48)
    at org.apache.uima.analysis_engine.impl.PrimitiveAnalysisEngine_impl.callAnalysisComponentProcess(PrimitiveAnalysisEngine_impl.java:411)
    at org.apache.uima.analysis_engine.impl.PrimitiveAnalysisEngine_impl.innerCall(PrimitiveAnalysisEngine_impl.java:329)
    at org.apache.uima.analysis_engine.impl.PrimitiveAnalysisEngine_impl.processAndOutputNewCASes(PrimitiveAnalysisEngine_impl.java:321)
    at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(ASB_impl.java:570)
    at org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.<init>(ASB_impl.java:412)
    at org.apache.uima.analysis_engine.asb.impl.ASB_impl.process(ASB_impl.java:344)
    at org.apache.uima.analysis_engine.impl.AggregateAnalysisEngine_impl.processAndOutputNewCASes(AggregateAnalysisEngine_impl.java:271)
    at org.apache.uima.analysis_engine.impl.AnalysisEngineImplBase.process(AnalysisEngineImplBase.java:269)
    at org.apache.uima.fit.pipeline.SimplePipeline.runPipeline(SimplePipeline.java:101)
    at de.tudarmstadt.ukp.dkpro.examples.stanfordcorecomponents.StanfordCoreComponents.main(StanfordCoreComponents.java:38)
reckart commented 5 years ago

Have you tried more, e.g. -Xmx4g? We run our unit tests with -Xmx6g

jhammen commented 5 years ago

Thank you for the advice, you are correct! Pipeline now runs after changing -Xmx2g to -Xmx6g