Open vitorpamplona opened 2 years ago
cowtowncoder
commented
2 years ago Quick note: almost certainly XML dataformat issue, not JAXB.
Thank you for providing tests, that's great!
Would be great to avoid using JAXB annotations if possible (that is, use Jackson ones) but this works as is too.
vitorpamplona
commented
2 years ago Would be great to avoid using JAXB annotations if possible (that is, use Jackson ones) but this works as is too.
I don't have a choice. :(
The class structure + annotations are frozen. That's why I am using that MixIn as well.
cowtowncoder
commented
2 years ago Sorry, what I meant was that only the unit test (reproduction) used different annotations, triggering the same issue. Not that your production code would have to change. This has to do with project dependencies where JAXB/XML coupling is slightly tricky (we were able to eliminate it for non-test code to avoid forcing JAXB dependency for users who do not need it). Test dependency is ok if not ideal; just means that when releasing versions there is a dependency if so (JAXB annotations module has to be released first).
vitorpamplona
commented
2 years ago ohhh! makes sense.
Here's the class without JAXB, running into the same problem.
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonSubTypes;
import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.dataformat.xml.XmlMapper;
import org.testng.annotations.Test;
import java.io.IOException;
import java.util.List;
import static org.junit.Assert.assertEquals;
class ModelInfo {
@JsonProperty List<TypeInfo> typeInfo;
}
@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "type")
@JsonSubTypes({
@JsonSubTypes.Type(value = ClassInfo.class)
})
abstract class TypeInfo {
}
class ClassInfo extends TypeInfo {
@JsonProperty List<ClassInfoElement> element;
@JsonProperty String name;
}
class ClassInfoElement {
@JsonProperty BindingInfo binding;
}
class BindingInfo {
@JsonProperty String description;
}
public class JacksonXMLTests {
XmlMapper mapper = new XmlMapper().builder()
.defaultUseWrapper(false)
.build();
@Test
public void testTypeAfterOtherProperties() throws IOException {
String xml =
"<modelInfo xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\">\n" +
" <typeInfo name=\"MyName\" xsi:type=\"ClassInfo\">\n" +
" <element>\n" +
" <binding description=\"Test\"/>\n" +
" </element>\n" +
" </typeInfo>\n" +
"</modelInfo>";
ModelInfo m = mapper.readValue(xml, ModelInfo.class);
assertEquals("MyName", ((ClassInfo)m.typeInfo.get(0)).name);
assertEquals("Test", ((ClassInfo)m.typeInfo.get(0)).element.get(0).binding.description);
}
@Test
public void testTypeBeforeOtherProperties() throws IOException {
String xml =
"<modelInfo xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\">\n" +
" <typeInfo xsi:type=\"ClassInfo\" name=\"MyName\">\n" +
" <element>\n" +
" <binding description=\"Test\"/>\n" +
" </element>\n" +
" </typeInfo>\n" +
"</modelInfo>";
ModelInfo m = mapper.readValue(xml, ModelInfo.class);
assertEquals("MyName", ((ClassInfo)m.typeInfo.get(0)).name);
assertEquals("Test", ((ClassInfo)m.typeInfo.get(0)).element.get(0).binding.description);
}
}Simplifying further to:
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonSubTypes;
import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.dataformat.xml.XmlMapper;
import org.testng.annotations.Test;
import java.io.IOException;
import java.util.List;
import static org.junit.Assert.assertEquals;
@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "type")
@JsonSubTypes({
@JsonSubTypes.Type(value = ClassInfo.class)
})
abstract class TypeInfo {
}
class ClassInfo extends TypeInfo {
@JsonProperty List<ClassInfoElement> element;
@JsonProperty String name;
}
class ClassInfoElement {
@JsonProperty BindingInfo binding;
}
class BindingInfo {
@JsonProperty String description;
}
public class JacksonXMLTests {
XmlMapper mapper = new XmlMapper().builder().defaultUseWrapper(false).build();
@Test
public void testTypeAfterOtherProperties() throws IOException {
String xml =
" <typeInfo name=\"MyName\" type=\"ClassInfo\">\n" +
" <element>\n" +
" <binding description=\"Test\"/>\n" +
" </element>\n" +
" </typeInfo>";
TypeInfo m = mapper.readValue(xml, TypeInfo.class);
assertEquals("MyName", ((ClassInfo)m).name);
assertEquals("Test", ((ClassInfo)m).element.get(0).binding.description);
}
@Test
public void testTypeBeforeOtherProperties() throws IOException {
String xml =
" <typeInfo type=\"ClassInfo\" name=\"MyName\">\n" +
" <element>\n" +
" <binding description=\"Test\"/>\n" +
" </element>\n" +
" </typeInfo>";
TypeInfo m = mapper.readValue(xml, TypeInfo.class);
assertEquals("MyName", ((ClassInfo)m).name);
assertEquals("Test", ((ClassInfo)m).element.get(0).binding.description);
}
}Some additional learnings:
mapper.readValue(xml, ClassInfo.class)List<ClassInfoElement> element into ClassInfoElement element also makes it workClassInfoElement and not the immediate parent. With another child entity:
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonSubTypes;
import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.dataformat.xml.XmlMapper;
import org.testng.annotations.Test;
import java.io.IOException;
import java.util.List;
import static org.junit.Assert.assertEquals;
@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "type")
@JsonSubTypes({
@JsonSubTypes.Type(value = ClassInfo.class)
})
abstract class TypeInfo {
}
class ClassInfo extends TypeInfo {
@JsonProperty List<ClassInfoElement> element;
@JsonProperty String name;
}
class ClassInfoElement {
@JsonProperty BindingInfo binding;
}
class BindingInfo {
@JsonProperty BindingInfo2 binding2;
}
class BindingInfo2 {
@JsonProperty String description;
}
public class JacksonXMLTests {
XmlMapper mapper = new XmlMapper().builder().defaultUseWrapper(false).build();
@Test
public void testTypeAfterOtherProperties() throws IOException {
String xml =
" <typeInfo name=\"MyName\" type=\"ClassInfo\">\n" +
" <element>\n" +
" <binding>\n" +
" <binding2 description=\"Test\"/>\n" +
" </binding>\n" +
" </element>\n" +
" </typeInfo>";
TypeInfo m = mapper.readValue(xml, TypeInfo.class);
assertEquals("MyName", ((ClassInfo)m).name);
assertEquals("Test", ((ClassInfo)m).element.get(0).binding.binding2.description);
}
@Test
public void testTypeBeforeOtherProperties() throws IOException {
String xml =
" <typeInfo type=\"ClassInfo\" name=\"MyName\">\n" +
" <element>\n" +
" <binding>\n" +
" <binding2 description=\"Test\"/>\n" +
" </binding>\n" +
" </element>\n" +
" </typeInfo>";
TypeInfo m = mapper.readValue(xml, TypeInfo.class);
assertEquals("MyName", ((ClassInfo)m).name);
assertEquals("Test", ((ClassInfo)m).element.get(0).binding.binding2.description);
}
}Ok, I was able to reproduce the failure and added the failing test case as reproduction.
From exception it does look like there is a structural mismatch, which I suspect is due to buffering that is needed when the type id is NOT the first property being found (that is, buffering of non-type-id properties until type id is encountered and handled). That should not matter, of course.
cowtowncoder
commented
2 years ago As to the root cause and possible fix... I think the problem has to do with handling of the "unwrapped" collections; the state is probably not retained by buffering via TokenBuffer (it being format-agnostic).
Jackson 2.13 does allow construction of format-specific TokenBuffers which might help solving the problem (instances are constructed via DeserializationContext and XML module already providers its own implementations for other reasons) but I don't remember if I ever figured out what information would need to be retained and how, unfortunately.
So on plus side I think this would be fixable, but downside is that I probably won't have time to spend on digging deep enough to do so.
But I'll keep the issue in mind in case something pops up.
The way I have used to dig deeper in the past has been to uncomment debug printing in FromXmlParser (and underlying XmlTokenStream) to see how XML elements and attributes get translated into equivalent JsonToken tokens; and how this varies across test cases. It's quite involved so I don't expect anyone else to do that, but it does show exactly what is happening, fwtw.
teodord
commented
1 year ago I came across what I think is the same issue. In my case, the object model hierarchy is deeper and the same XML node is actually containing itself. When appearing deeper, the XML node has slightly different attributes and the xsi:type ends up in a different position. Unfortunately, I consume XML that is produced in a different system and I cannot sanitize it before attempting to load it with Jackson. Can you suggest any workaround?
vitorpamplona
commented
1 year ago I had to hack the XML with a replaceAll using Regex before parsing it:
From: <typeInfo ([^>]*) xsi:type="ClassInfo">
To: <typeInfo xsi:type="ClassInfo" $1>
teodord
commented
1 year ago I am adding a different exception to this tracker hoping that it would help solve this issue for all cases.
In addition to the unrecognized field exception already mentioned above, I also have a test case where the following exception is thrown, the cause being probably the same, which is the position of the xsi:type attribute in the polymorphic node.
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of java.lang.String[]: no String-argument constructor/factory method to deserialize from String value ('pulp') at [Source: (StringReader); line: 3, column: 17] (through reference chain: Apple["parent"]->Apple["part"]) at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67) at com.fasterxml.jackson.databind.DeserializationContext.reportBadDefinition(DeserializationContext.java:1904) at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:400) at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1344) at com.fasterxml.jackson.databind.deser.std.StdDeserializer._deserializeFromString(StdDeserializer.java:311) at com.fasterxml.jackson.databind.deser.std.StringArrayDeserializer.handleNonArray(StringArrayDeserializer.java:317) at com.fasterxml.jackson.databind.deser.std.StringArrayDeserializer.deserialize(StringArrayDeserializer.java:141) at com.fasterxml.jackson.databind.deser.std.StringArrayDeserializer.deserialize(StringArrayDeserializer.java:25) at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129) at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:313) at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:214) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:186) at com.fasterxml.jackson.dataformat.xml.deser.WrapperHandlingDeserializer.deserialize(WrapperHandlingDeserializer.java:122) at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:144) at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:110) at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:263) at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:138) at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:313) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:176) at com.fasterxml.jackson.dataformat.xml.deser.WrapperHandlingDeserializer.deserialize(WrapperHandlingDeserializer.java:122) at com.fasterxml.jackson.dataformat.xml.deser.XmlDeserializationContext.readRootValue(XmlDeserializationContext.java:91) at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4674) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3629) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3597) at net.sf.jasperreports.util.JacksonUtil.loadXml(JacksonUtil.java:193) ... 8 more
I have attached the two Java source files and the XML snippet that causes the error.
cowtowncoder
commented
1 year ago @teodord I appreciate your trying to help here but the issue here is not a potential reproductions -- the problem is almost certainly related to buffering of content that is required when the type id is not the first thing read. I am not sure these are caused by the same root issue either (except in a very general sense).
@vitorpamplona Ok here is a case of buffering being applied. One thing that you can see from deserializer (if you had custom one for example) is that the JsonParser instance given is not FromXmlParser but implementation by TokenBuffer (inner class). The problem being that the information retained is not exactly the same as what is read from input stream (because XML reader used will apply some conversions to make it look like stream of JsonTokens that JSON-backed decoder would provide).
This very specific structure fails to parse a correct XML when the
xsi:typeproperty is placed after other fields in the XML. That's the only difference between the test cases.See below:
Running this with:
Yields a pass on
testTypeBeforeOtherPropertiesand a fail ontestTypeAfterOtherPropertieswith the following stack:I am not sure if this is a JAXB Module issue or a DataFormat issue. I hope this is the right place.