Inconsistent serialization / deserialization

[arlyon]

Hello. I have been using this to deserialise incoming data but have hit a snag. Consider these cases.

class MyClass {
   private Optional<String> about;
}

{ "about": null }
{}
{ "about": "Hello World"}

During deserialization, the first case maps to a Optional.empty(), while the second maps to null and third maps to Optional.of("Hello World"). This is great; I can assume that if the field is non-null it was included in the json and can respond accordingly.

During serialization, however, both null and Optional.empty() map to {"about": null} which breaks the expectation that composing the two operations preserves the identity, namely that in its current form serialize(deserialize(data)) != data.

What is the reason for this? Is there a way to configure it to respect the identity?

EDIT:

You can preserve serialization -> deserialization by marking the types you want to preserve identity on with the annotation @JsonInclude(NON_ABSENT). Classes that do not feature this annotation will not have identity preserved.

[cowtowncoder]

@arlyon This is pretty complicated setup to handle and unfortunately handling is not optimal. Part of the reason is that support was built over time and round-trip consistency (serialize->deserialize retains original value) is not fully working. And since there is now established behavior, changing is not necessarily possible without breaking some existing usage.

There is also specific limitation in that "absent" value is handled differently depending on whether property is set via Field, Method, or constructor parameter. In first two cases missing value is not set at all -- only properties for which JSON has value are considered, others are left as is. But with constructor parameters there is a way to specify "absent" value to use (because constructor parameter must have some value). Because of this, for field-valued Optional properties it makes sense to use Java level initialization.

Also: good note on @JsonIncude(NON_ABSENT). This is indeed necessary since serializers cannot, on their own, decide NOT to serialize a value (this is due the way things are modeled; serializers do not output property name, so by the time they are invoked, a value MUST be written as JSON) But various (mostly annotation-based) filtering mechanisms do allow omitting output of a property.

This is a known problem area, for what that is worth. I was hoping to at least test and document actual behavior first, for 2.14, and giving timing, try to see if handling could be improved for 2.15, possibly with some on/off configuration features to retain backwards compatibility. One specific area considered for changes is the choice of getNullValue() and getAbsentValue() for Optional deserializer. Right now two are the same but they probably should not be. As I mentioned earlier, absent value only affects Constructor-parameter -based properties, however.

[arlyon]

Thank you for the excellent and detailed reply, and for your time contributing to this library. Very insightful.

Happy to close this now. All the best,

Alex

[cowtowncoder]

Thank you @arlyon! I do hope to improve situation here in 2.15 and your thoughtful discussion helps here, to know it is something users would find useful too. Within constraints backwards-compatibility. I just wanted it to be known that I think you are absolutely right wrt confusing aspects of handling -- and that round-trip handling is a very important goal, sometimes difficult to achieve (especially incrementally if not working initially), but goal nonetheless.