Cannot use @JsonbCreator with absent fields

[emattheis]

In my experience, it is quite common to have JSON documents whose properties are present/absent depending on the scenario in which they are used. For example, the following two JSON documents might both represent a user:

{
  "firstName": "John",
  "lastName": "Doe"
}

{
  "firstName": "John",
  "middleInitial": "Q",
  "lastName": "Public"
}

I can easily map these documents to a Java class using the classic JavaBean pattern and JSON-B will be happy to deserialize either of the two documents above:

public class User {
    private String firstName;

    private String middleInitial;

    private String lastName;

    public String getFirstName() {
        return firstName;
    }

    public void setFirstName(String firstName) {
        this.firstName = firstName;
    }

    public String getMiddleInitial() {
        return middleInitial;
    }

    public void setMiddleInitial(String middleInitial) {
        this.middleInitial = middleInitial;
    }

    public String getLastName() {
        return lastName;
    }

    public void setLastName(String lastName) {
        this.lastName = lastName;
    }
}

However, if I want to map these documents to a Java class using an immutable pattern leveraging the @JsonbCreator annotation, it is disallowed by the spec:

import javax.json.bind.annotation.JsonbCreator;
import javax.json.bind.annotation.JsonbProperty;

public class User {
    private final String firstName;

    private final String middleInitial;

    private final String lastName;

    @JsonbCreator
    public User(@JsonbProperty("firstName") String firstName,
                @JsonbProperty("middleInitial") String middleInitial,
                @JsonbProperty("lastName") String lastName) {
        this.firstName = firstName;
        this.middleInitial = middleInitial;
        this.lastName = lastName;
    }

    public String getFirstName() {
        return firstName;
    }

    public String getMiddleInitial() {
        return middleInitial;
    }

    public String getLastName() {
        return lastName;
    }
}

Section 4.5 say:

In case a field required for a parameter mapping doesn’t exist in JSON document, JsonbException MUST be thrown.

In my opinion, usage of @JsonbCreator should be compatible with any document that can otherwise be deserialized by the default mapping methods, so this spec requirement is at odds with Section 3.14, which says:

The deserialization operation of a property absent in JSON document MUST not set the value of the field, the setter (if available) MUST not be called, and thus original value of the field MUST be preserved.

[nmatt]

@rmannibucau: This is not about validation. it's about the case where both the property being absent and and the property being explicitly null in the JSON are valid, but have different semantics (*). This is straightforward with setters, and therefore arguably should be made straightforward with constructores.

(*) That may seem like a poor design choice, but can come up due to API evolution, among other things. For example, version 2 of the API may introduce a new mandatory property, whose value was previously merely implied. To continue supporting existing clients, the previously implied value is used as the default value when the property is absent in the JSON (in the Java class, by initializing the corresponding field with that value). The setter of the new property doesn't allow null because the property is mandatory (= the getter always returns a non-null value). In version 3, a need comes up to make the value represented by the property optional. The setter is therefore relaxed to allow null. JSON sent by version 1 clients (i.e. with the property absent) continues to map to the specific default value, whose semantics is different from the new null value.

[rmannibucau]

@nmatt hmm, not really. null presence or absence is not something the mapper should cover because otherwise you must also cover bigdecimal as string or number difference etc. All that is impossible without adding Jsonb specific inputs you don't really need. Also note that the setter option already does not work since null can be skipped by the impl (should actually) so it can not be called if null is there. So really this feature is not a feature of the spec as of today (and I think it is fine). At the end it is only about valued validation which as explained is not relevant so we can drop it with the backward compat toggle.

[nmatt]

@rmannibucau: The JSON-B spec at https://download.oracle.com/otndocs/jcp/json_b-1-pr-spec/ (linked from https://javaee.github.io/jsonb-spec/download.html) requires that the distinction between absent and null be preserved (section 3.14.1). Has this been changed? Where can I get the current spec? (never mind, found it)

EDIT: This is also still true in the 2.0.0 spec, same section ("Null Java field").

[rmannibucau]

@nmatt you are right for the setter (for the field it is way more ambiguous since the default can be detected being or not null and then the set be skipped) but it also confirms my points: it is schema validation (attribute presence) whch is not JSON-B about nor a feature we wanted to bundle in 1.0. If it is really desired JsonValue injection support is there and enables all needed validation functionally. Don't get me wrong, I'm not saying validation is not something jakarta should cover but not in jsonb "core" and not just one of the dozen of validations rules jsonschema covers.

[nmatt]

@rmannibucau: I disagree that this is about validation. This is about the conceptual data model: Are absent properties and null properties equivalent (same logical value) or not when deserializing JSON? The existing spec says that they are not interchangeable. The existing @JsonbCreator does not support absent values (consistent with the fact that they are not interchangeable). Adding support for absent values, but conflating them with null values, on the one hand brings it closer to the setter capabilities, but on the other hand creates an inconsistency with the setters regarding the data model for deserialization. In my mind it would be better to maintain a consistent model.

[rmannibucau]

@nmatt

Are absent properties and null properties equivalent (same logical value) or not when deserializing JSON?

This is the point, for JSON yes. I know you can use the presence to do anything different but for JSON (and more widely JSON ecosystem, including the most used, javascript) null or absence is the same.

Once again, I'm not saying you must not be able to detect it is there or not but:

1. You are able to do
2. It is about shape validation (this is why i mention jsonschema which is likely the most known solution for that) and not about mapping itself

If you don't agree the issue is quite immediate:

public class Foo { public String bar = "dummy"; }

if you get {"bar":null} and {} you dont have the same model whereas it is functionally equivalent in terms of data (it is not in terms of schema but once again JSON-B is not, as of today, about the schema at all)

[nmatt]

@rmannibucau: I don't want to draw this argument out unnecessarily, but to make some final points: It is not about shape validation if both absent and null are valid within the same schema but have different semantics. By "data model" I didn't mean "schema", but the generic meta-model of (arbitrary) JSON data; no schemata involved. My point is you are choosing a different meta-model now for constructors than what JSON-B has for setters. It's also not correct to say that Javascript doesn't distinguish between absent and null: an absent property is undefined in Javascript, not null, and thus {"bar":null}.bar === {}.bar evaluates to false.

[rmannibucau]

@nmatt you are picky but it does not change the point, in a java object null or absence is the same, nothing to discuss there. In javascript it is the same too until you validate the shape (schema validation) where you check with undefined test or hasOwnProperty method. So it is correct to say the difference between an explicit null and an absence is only about the schema, not the data. Once again, I see your use case and think at some point we must cover it but it is not in the scope of this issue nor in JSON-B scope (yet).

[erik-mattheis-snhu]

Glad to see some interest here even though I've largely moved on and given up on JSON-B 😛

The problem I originally ran into is that the specification and implementation force validation into the picture by requiring restrictive behavior when using a creator that isn't required in the setter approach. With the setter approach, I can tell if the field was present in the JSON based on whether or not setter was called (assuming an implementation is required to call the setter when the value is explicitly null - I'm not going to check that right now). Of course, I can't do that in a creator because all the values are passed at once, so I was seeking flexibility there.

This is undeniably a useful thing that many people need in order to adopt JSON-B.

Arguing about whether it is validation or not is pointless - that is water under the bridge.

The fact that Java has no distinction between null and undefined is also immaterial. JSON certainly supports payloads that make a semantic distinction between properties that are absent and properties that are present with a null value. This is not picky, this is the real world with messy polyglot implementations.

This is also a completely solved problem in other JSON Java libraries, so it remains a shame to see JSON-B held back from more widespread adoption by this continuous debate.

[rmannibucau]

Just a reminder it is solved in jsonb since v1, jsonvalues can be injected and marshalled as in any other json lib...

[emattheis]

@rmannibucau I think you understand what I meant.

Nobody in this thread is arguing about the functionality that we know works. The inability to use JSON-B to bind JSON to immutable types in Java is a shame. There is no technical reason this can't be solved.

[Verdent]

@emattheis @nmatt @rmannibucau @m0mus Thank you guys for your feedback on the issue here :-) I will prepare PR with the proposed changes and I will try to address your arguments there.