Bug when XML has \n in it!

victornoel commented 7 years ago

Hi,

If I execute a transformation from XML to JSON with the following:

        final JsonXMLConfig config = new JsonXMLConfigBuilder().virtualRoot(METADATAS).build();

        String input = "<metadatas>\n  <?xml-multiple metadata?>\n</metadatas>";

        System.out.println("XML : " + input);

        XMLEventReader reader = XMLInputFactory.newInstance()
                .createXMLEventReader(new ByteArrayInputStream(input.getBytes()));

        final OutputStream output = new ByteArrayOutputStream();
        /*
         * Create writer (JSON).
         */
        XMLEventWriter writer = new JsonXMLOutputFactory(config).createXMLEventWriter(output);

        /*
         * Copy events from reader to writer.
         */
        writer.add(reader);

        System.out.println("XML --> JSON : " + output.toString());

I get the following error:

java.lang.IllegalStateException: Nesting problem.
    at com.google.gson.stream.JsonWriter.beforeName(JsonWriter.java:616)
    at com.google.gson.stream.JsonWriter.writeDeferredName(JsonWriter.java:401)
    at com.google.gson.stream.JsonWriter.value(JsonWriter.java:417)
    at de.odysseus.staxon.json.stream.gson.GsonStreamTarget.value(GsonStreamTarget.java:61)
    at de.odysseus.staxon.json.stream.util.StreamTargetDelegate.value(StreamTargetDelegate.java:52)
    at de.odysseus.staxon.json.JsonXMLStreamWriter.writeEndElementTag(JsonXMLStreamWriter.java:203)
    at de.odysseus.staxon.base.AbstractXMLStreamWriter.writeEndElement(AbstractXMLStreamWriter.java:217)
    at de.odysseus.staxon.event.SimpleXMLEventWriter.add(SimpleXMLEventWriter.java:60)
    at de.odysseus.staxon.event.SimpleXMLEventWriter.add(SimpleXMLEventWriter.java:119)
    at org.ow2.petals.binding.rest.exchange.JSONHelperTest.test(JSONHelperTest.java:181)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)

But if I use as input instead: <metadatas><?xml-multiple metadata?></metadatas> then it works without problem!

victornoel commented 7 years ago

Hm, but maybe it is a bug in gson actually…

victornoel commented 7 years ago

but I don't see how, since the only thing that differs is the XML which gson does know nothing about…

victornoel commented 7 years ago

I am using gson 2.8.0 but it appears also with the version in staxon-gson.

victornoel commented 7 years ago

Note the bug is not visible when using the default json serializer

beckchr commented 7 years ago

Did you try jackson or staxon's built-in json support instead of gson?

victornoel commented 7 years ago

let me try with jackson, but with built-in it works

victornoel commented 7 years ago

it does work with jackson, so the problem only appears with gson.

But I don't see how adding carriage return to the XML should be visible to gson, shouldn't staxon abstract that?

victornoel commented 7 years ago

ah no sorry, I didn't try, I made a mistake… I will tell you

victornoel commented 7 years ago

Here is a stacktrace with Jackson…

javax.xml.transform.TransformerException: org.xml.sax.SAXException: Cannot write end element: metadatas
javax.xml.stream.XMLStreamException: Cannot write end element: metadatas
    at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:758)
    at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:359)
    at org.ow2.petals.binding.rest.exchange.JSONHelper.convertXMLToJSON(JSONHelper.java:85)
    at org.ow2.petals.binding.rest.exchange.JSONHelperTest.carriageReturnInXML(JSONHelperTest.java:338)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: org.xml.sax.SAXException: Cannot write end element: metadatas
javax.xml.stream.XMLStreamException: Cannot write end element: metadatas
    at com.sun.org.apache.xalan.internal.xsltc.trax.SAX2StAXStreamWriter.endElement(SAX2StAXStreamWriter.java:209)
    at com.sun.org.apache.xml.internal.serializer.ToXMLSAXHandler.endElement(ToXMLSAXHandler.java:265)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2973)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:117)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
    at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transformIdentity(TransformerImpl.java:650)
    at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:746)
    ... 26 more
Caused by: javax.xml.stream.XMLStreamException: Cannot write end element: metadatas
    at de.odysseus.staxon.json.JsonXMLStreamWriter.writeEndElementTag(JsonXMLStreamWriter.java:214)
    at de.odysseus.staxon.base.AbstractXMLStreamWriter.writeEndElement(AbstractXMLStreamWriter.java:217)
    at com.sun.org.apache.xalan.internal.xsltc.trax.SAX2StAXStreamWriter.endElement(SAX2StAXStreamWriter.java:205)
    ... 39 more
Caused by: com.fasterxml.jackson.core.JsonGenerationException: Can not write a field name, expecting a value
    at com.fasterxml.jackson.core.JsonGenerator._reportError(JsonGenerator.java:1897)
    at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeFieldName(UTF8JsonGenerator.java:185)
    at de.odysseus.staxon.json.stream.jackson.JacksonStreamTarget.name(JacksonStreamTarget.java:45)
    at de.odysseus.staxon.json.JsonXMLStreamWriter.writeEndElementTag(JsonXMLStreamWriter.java:201)
    ... 41 more
---------
org.xml.sax.SAXException: Cannot write end element: metadatas
javax.xml.stream.XMLStreamException: Cannot write end element: metadatas
    at com.sun.org.apache.xalan.internal.xsltc.trax.SAX2StAXStreamWriter.endElement(SAX2StAXStreamWriter.java:209)
    at com.sun.org.apache.xml.internal.serializer.ToXMLSAXHandler.endElement(ToXMLSAXHandler.java:265)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2973)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:117)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
    at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transformIdentity(TransformerImpl.java:650)
    at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:746)
    at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:359)
    at org.ow2.petals.binding.rest.exchange.JSONHelper.convertXMLToJSON(JSONHelper.java:85)
    at org.ow2.petals.binding.rest.exchange.JSONHelperTest.carriageReturnInXML(JSONHelperTest.java:338)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: javax.xml.stream.XMLStreamException: Cannot write end element: metadatas
    at de.odysseus.staxon.json.JsonXMLStreamWriter.writeEndElementTag(JsonXMLStreamWriter.java:214)
    at de.odysseus.staxon.base.AbstractXMLStreamWriter.writeEndElement(AbstractXMLStreamWriter.java:217)
    at com.sun.org.apache.xalan.internal.xsltc.trax.SAX2StAXStreamWriter.endElement(SAX2StAXStreamWriter.java:205)
    ... 39 more
Caused by: com.fasterxml.jackson.core.JsonGenerationException: Can not write a field name, expecting a value
    at com.fasterxml.jackson.core.JsonGenerator._reportError(JsonGenerator.java:1897)
    at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeFieldName(UTF8JsonGenerator.java:185)
    at de.odysseus.staxon.json.stream.jackson.JacksonStreamTarget.name(JacksonStreamTarget.java:45)
    at de.odysseus.staxon.json.JsonXMLStreamWriter.writeEndElementTag(JsonXMLStreamWriter.java:201)
    ... 41 more
---------
javax.xml.stream.XMLStreamException: Cannot write end element: metadatas
    at de.odysseus.staxon.json.JsonXMLStreamWriter.writeEndElementTag(JsonXMLStreamWriter.java:214)
    at de.odysseus.staxon.base.AbstractXMLStreamWriter.writeEndElement(AbstractXMLStreamWriter.java:217)
    at com.sun.org.apache.xalan.internal.xsltc.trax.SAX2StAXStreamWriter.endElement(SAX2StAXStreamWriter.java:205)
    at com.sun.org.apache.xml.internal.serializer.ToXMLSAXHandler.endElement(ToXMLSAXHandler.java:265)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2973)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:117)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
    at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transformIdentity(TransformerImpl.java:650)
    at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:746)
    at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:359)
    at org.ow2.petals.binding.rest.exchange.JSONHelper.convertXMLToJSON(JSONHelper.java:85)
    at org.ow2.petals.binding.rest.exchange.JSONHelperTest.carriageReturnInXML(JSONHelperTest.java:338)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: com.fasterxml.jackson.core.JsonGenerationException: Can not write a field name, expecting a value
    at com.fasterxml.jackson.core.JsonGenerator._reportError(JsonGenerator.java:1897)
    at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeFieldName(UTF8JsonGenerator.java:185)
    at de.odysseus.staxon.json.stream.jackson.JacksonStreamTarget.name(JacksonStreamTarget.java:45)
    at de.odysseus.staxon.json.JsonXMLStreamWriter.writeEndElementTag(JsonXMLStreamWriter.java:201)
    ... 41 more

beckchr commented 7 years ago

Okay thanks. Will have to investigate this...

victornoel commented 7 years ago

@beckchr anything I can do to help narrow the source of this problem? If you have ideas where I could look, I can start digging :)

beckchr commented 7 years ago

The problem is, that the event reader passes whitespace as character event to the writer. I'm not sure how to fix this without unwanted side effects (ignoring whitespace when I shouldn't).

For now, the easiest thing may be to ignore whitespace completely. You can do this by adding an EventFilter to your event reader like this:

reader = XMLInputFactory.newInstance().createFilteredReader(reader, new EventFilter() {
    @Override
    public boolean accept(XMLEvent event) {
      return !event.isCharacters() || !event.asCharacters().isWhiteSpace();
    }
});

Hope that helps.

victornoel commented 7 years ago

@beckchr thanks for the idea!

beckchr / staxon

Bug when XML has \n in it! #38