search

rahuldave / semflow

Semantic Pipeline for ADS
6 stars 3 forks source link

Java heap space #8

Open DougBurke opened 13 years ago

DougBurke commented 13 years ago

newmast/mast_obsvload.py failed during an IUE run with

../mast-rdf/iue/obscore.iue.psv.swp44709mxlo.rdf
LOADING ../mast-rdf/iue/obscore.iue.psv.swp44709mxlo.rdf
Code:  500
Error code:  500
Traceback (most recent call last):
  File "newmast/mast_obsvload.py", line 42, in <module>
    c.postfile(filename, context)
  File "/Users/dburke/semweb/ADS/ads-stuff/semflow/pysesame.py", line 204, in postfile
    res=self.postdata(data, context, method)
  File "/Users/dburke/semweb/ADS/ads-stuff/semflow/pysesame.py", line 188, in postdata
    raise ValueError(e.code)
ValueError: 500

which appears to be because sesame has run out of stack space since we get from the java -jar start.jar process:


2011-07-14 12:45:29.507:WARN::/openrdf-sesame/repositories/testads8/statements
java.lang.OutOfMemoryError: Java heap space
    at java.util.LinkedList.listIterator(LinkedList.java:667)
    at java.util.AbstractList.listIterator(AbstractList.java:284)
    at java.util.AbstractSequentialList.iterator(AbstractSequentialList.java:222)
    at org.openrdf.sail.nativerdf.btree.BTree$Node.notifyNodeSplit(BTree.java:1738)
    at org.openrdf.sail.nativerdf.btree.BTree$Node.splitAndInsert(BTree.java:1664)
    at org.openrdf.sail.nativerdf.btree.BTree.insertInNode(BTree.java:919)
    at org.openrdf.sail.nativerdf.btree.BTree.insertInTree(BTree.java:889)
    at org.openrdf.sail.nativerdf.btree.BTree.insertInTree(BTree.java:895)
    at org.openrdf.sail.nativerdf.btree.BTree.insertInTree(BTree.java:895)
    at org.openrdf.sail.nativerdf.btree.BTree.insertInTree(BTree.java:895)
    at org.openrdf.sail.nativerdf.btree.BTree.insert(BTree.java:836)
    at org.openrdf.sail.nativerdf.TripleStore.storeTriple(TripleStore.java:720)
    at org.openrdf.sail.nativerdf.NativeStoreConnection.addStatement(NativeStoreConnection.java:393)
    at org.openrdf.sail.nativerdf.NativeStoreConnection.addStatementInternal(NativeStoreConnection.java:346)
    at org.openrdf.sail.helpers.SailConnectionBase.addStatement(SailConnectionBase.java:331)
    at org.openrdf.repository.sail.SailRepositoryConnection.addWithoutCommit(SailRepositoryConnection.java:236)
    at org.openrdf.repository.base.RepositoryConnectionBase.add(RepositoryConnectionBase.java:494)
    at org.openrdf.repository.util.RDFInserter.handleStatement(RDFInserter.java:193)
    at org.openrdf.rio.rdfxml.RDFXMLParser.reportStatement(RDFXMLParser.java:1010)
    at org.openrdf.rio.rdfxml.RDFXMLParser.processPropertyElt(RDFXMLParser.java:684)
    at org.openrdf.rio.rdfxml.RDFXMLParser.emptyElement(RDFXMLParser.java:378)
    at org.openrdf.rio.rdfxml.SAXFilter.endElement(SAXFilter.java:359)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:601)
    at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:180)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2755)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:511)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
kayebohemier commented 13 years ago

The OutOfMemoryError is a known problem, and I think the rest of us have had it at some point (but before we started tracking issues). It can be fixed by explicitly providing the heap size in the command:

java -Xmxn####m -Xmx####m -jar MyApp.jar

This is what the java man file says about these commands:

-Xmsn :: Specifies the initial size of the memory allocation pool. This value must be a multiple of 1024 greater than 1 MB. Append the letter k or K to indicate kilobytes, the letter m or M to indicate megabytes, the letter g or G to indicate gigabytes, or the letter t or T to indicate terabytes. The default value is 2MB. Examples:

                       -Xms6291456
                       -Xms6144k
                       -Xms6m

-Xmxn :: Specifies the maximum size, in bytes, of the memory allocation pool. This value must be a multiple of 1024 greater than 2 MB. Append the letter k or K to indicate kilobytes, the letter m or M to indicate megabytes, the letter g or G to indicate gigabytes, or the letter t or T to indicate terabytes. The default value is 64MB. Examples:

                       -Xmx83886080
                       -Xmx81920k

I'm not sure if there's a way to specify a default maximum heap size without explicitly stating it in the start command.

DougBurke commented 13 years ago

Kayleigh, Yeah, you need to use both the -Xms/-Xmx flags. I believe the default is 64mb (oh yeah, it says it above).

This happened on obsid 435 out of 68020 so it could be fun getting the correct value.