OutOfMemoryError (Head space)

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Get a v21s format Catalog with around 100,000 product entries
2. Run the command-line: java -Xmx2056m -jar jonix-2.0.jar ONIX.xml output.csv 
v21s
3. It throws the error after a while (74346 entries processes)

What is the expected output? What do you see instead?
Library as it is consumes too much Heap, this a major flaw, at the moment its 
the bigger the size the more heap it consumes,

What version of the product are you using? On what operating system?
2.0

Please provide any additional information below.
Maybe reading and writing in chunks ? a command-line parameter that defines the 
chunk size ? incremental writes etc?

Original issue reported on code.google.com by tigaro...@gmail.com on 20 Mar 2013 at 12:44

[GoogleCodeExporter]

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at com.sun.org.apache.xerces.internal.util.XMLStringBuffer.append(XMLStringBuffer.java:205)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.refresh(XMLDocumentScannerImpl.java:1522)
    at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.invokeListeners(XMLEntityScanner.java:2070)
    at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.skipChar(XMLEntityScanner.java:1415)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2793)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:511)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
    at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
    at org.jonix.JonixParser.parse(JonixParser.java:197)
    at org.jonix.JonixScanner.doScan(JonixScanner.java:154)
    at org.jonix.JonixScanner.scan(JonixScanner.java:94)
    at org.jonix.JonixFilesScanner.scanFileList(JonixFilesScanner.java:112)
    at org.jonix.JonixFilesScanner.scanFolder(JonixFilesScanner.java:94)
    at org.jonix.JonixFilesScanner.scanFolder(JonixFilesScanner.java:99)
    at org.jonix.Jonix.main(Jonix.java:56)

Original comment by tigaro...@gmail.com on 20 Mar 2013 at 12:45

[GoogleCodeExporter]

I had a similar issue using Jonix 1.0 whilst processing a large set of Onix 
files.  In the OOM dump it seemed that the parser was building a huge string as 
it was reading the file.

We corrected this be adding a xerces jar to the project which was then used in 
preference to the internal xerces parser.  Memory usage was then much more 
controlled and allowed the processing of a large set of files.

    <dependency>
        <groupId>xerces</groupId>
        <artifactId>xerces</artifactId>
        <version>2.4.0</version>
        <type>jar</type>
        <scope>compile</scope>
    </dependency>

Original comment by wlayt...@gmail.com on 10 Oct 2013 at 11:49