search

FasterXML / jackson-dataformats-binary

Uber-project for standard Jackson binary format backends: avro, cbor, ion, protobuf, smile
Apache License 2.0
314 stars 136 forks source link

[Help needed] Fix custom null serialization #182

Closed marcospassos closed 2 weeks ago

marcospassos commented 5 years ago

@cowtowncoder implementing a null custom serialization for an internal library, I stumbled upon a bug that prevents serialized null from being deserialized (which should affect other packages, such as JSR-353). For now, this PR only includes a test that reproduces the problem.

Error message:

com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input

Could you give me some guidance on how to fix it?

cowtowncoder commented 5 years ago

It'd be good to maybe file issue separately, link to/from this PR. There's one small problem with custom deserializer, not sure if it matters.

marcospassos commented 5 years ago

It'd be good to maybe file issue separately, link to/from this PR.

I'll open a PR.

I just pushed a new commit, but it doesn't seem to fix the issue.

marcospassos commented 5 years ago

Additional info: the same does not happen when serializing/deserializing to/from JSON.

cowtowncoder commented 5 years ago

Ok. Thank you for verifying.

I think this could be related to one oddity wrt Avro handling of nulls: unless null value is part of union, it "does not exist": that is, it does not take any space. Value of a field with only null type is always null, and nothing is encoded. This is fine in theory, but presents unique problem at root-level reader: it is not easy to know if stream has ended or not -- just because there are no bytes left does not necessarily mean we are done as null may still be implicitly available! (I had to fix one bug earlier related to this).

marcospassos commented 5 years ago

Dammit, now I see why I couldn't find an easy way to fix it myself without consuming your limited time. But it seems a critical bug to fix, as it can break things seriously. Again, let me know if I can help you fix it.

cowtowncoder commented 5 years ago

@marcospassos Unfortunately I do not really have time to work on this now.

cowtowncoder commented 2 weeks ago

I don't think this is active any longer; closing (may be re-filed of course)