search

2my / test-data-control

Automatically exported from code.google.com/p/test-data-control
Apache License 2.0
0 stars 0 forks source link

Json (+Xml) streaming #6

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
Jettison reads up entire string to make JSONObject, this leads to memory 
problems. Also dbUnit will exhaust heap when exporting large tables, even in 
"streaming" mode (see test failure below).
I want to be able to stream json, and possibly xml, so that I don't run out of 
space (until harddrive is full)

Some links that may help:

* xml - Using Jettison to go from JSON to DOM in Java:
http://stackoverflow.com/questions/854344/using-jettison-to-go-from-json-to-dom-
in-java
* Xml Reader To Writer : XMLStreamWriter « XML « Java
http://www.java2s.com/Code/Java/XML/XmlReaderToWriter.htm
* XMLEventReaderToContentHandler (StAX Utils API)
http://www.jarvana.com/jarvana/view/net/java/dev/stax-utils/stax-utils/20060502/
stax-utils-20060502-javadoc.jar!/javanet/staxutils/XMLEventReaderToContentHandle
r.html
* XMLStreamReaderToContentHandler
http://xfire.codehaus.org/maven/xfire/apidocs/org/codehaus/xfire/xmlbeans/util/X
MLStreamReaderToContentHandler.html

Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 23.633 sec <<< 
FAILURE!
verify_stream2FlatXml[1](no.antares.dbunit.DbWrapperTest)  Time elapsed: 17.888 
sec  <<< ERROR!
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2882)
    at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
    at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
    at java.lang.StringBuffer.append(StringBuffer.java:224)
    at java.io.StringWriter.write(StringWriter.java:84)
    at org.dbunit.util.xml.XmlWriter.openElement(XmlWriter.java:274)
    at org.dbunit.util.xml.XmlWriter.writeElement(XmlWriter.java:242)
    at org.dbunit.dataset.xml.FlatXmlWriter.row(FlatXmlWriter.java:196)
    at org.dbunit.dataset.stream.DataSetProducerAdapter.produce(DataSetProducerAdapter.java:104)
    at org.dbunit.dataset.xml.FlatXmlWriter.write(FlatXmlWriter.java:124)
    at org.dbunit.dataset.xml.FlatXmlDataSet.write(FlatXmlDataSet.java:405)
    at org.dbunit.dataset.xml.FlatXmlDataSet.write(FlatXmlDataSet.java:388)
    at no.antares.dbunit.DbWrapper.stream2FlatXml(DbWrapper.scala:113)

Original issue reported on code.google.com by t...@antares.no on 24 Jun 2011 at 11:00