[BUG][Java/spring] Incorrect Handling of Nullable Fields in OpenAPI Generator

Kavan72 commented 11 months ago

Bug Report Checklist

[x] Have you provided a full/minimal spec to reproduce the issue?
[x] Have you validated the input using an OpenAPI validator (example)?
[x] Have you tested with the latest master to confirm the issue still exists?
[x] Have you searched for related issues/PRs?
[x] What's the actual output vs expected output?
[ ] [Optional] Sponsorship to speed up the bug fix or feature request (example)

Description

Essentially, I have one endpoint to fetch user data (see below). What I'm attempting to achieve is to make phoneNumber optional, or set it as nullable: true. Therefore, I have two options: 1) Omit mentioning that field in the required section. 2) Use nullable: true at the field level.

In addition, I have a method that converts a UserDTO (database object) to a RestUser (generated model). Take a look at this method:

public static RestUser toRestUser(UserDTO userDTO) {

    return new RestUser()
        .id(userDTO.getId())
        .name(userDTO.getName())
        .phoneNumber(userDTO.getPhoneNumber())
        .email(userDTO.getEmailAddress());
}

1) If I don't specify that field as required in Swagger, the generator will create that field with Optional.of. At that point, if my value is null, it will throw a NullPointerException. In my opinion, we should use Optional.ofNullable for non-required fields. 2) If I use nullable: true on the field, I get a completely different value. If I pass a value to a nullable field, I receive this below result.

Actual output

However, this is incorrect why getting "present": true; the corrected value should be null if i didn't pass the value.

{
  "id": 1,
  "name": "John",
  "phoneNumber": {
    "present": true
  },
  "mail": "test@gmail.com"
}

Expected output

{
  "id": 1,
  "name": "John",
  "phoneNumber": null,
  "mail": "test@gmail.com"
}

openapi-generator version

7.2.0

OpenAPI declaration file content or url

openapi: 3.0.3
info:
  title: Backend
  version: 1.0.0

paths:
  /users:
    get:
      tags:
       - user
      summary: get.users
      description: get all users.
      responses:
        200:
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/User'

components:
  schemas:
    User:
      type: object
      properties:
        id: 
          type: integer
        name:
          type: string
        phoneNumber:
          type: integer
          format: int64
          nullable: true
        email:
          type: string

Kavan72 commented 10 months ago

@wing328 can you have a look on this issue ? I'm ready to contribute on this issue.

Moribund7 commented 10 months ago

@Kavan72 @wing328 I'm facing the same issue right now and I'm also willing to contribute here.

MelleD commented 6 months ago

If I don't specify that field as required in Swagger, the generator will create that field with Optional.of. At that point, if my value is null, it will throw a NullPointerException. In my opinion, we should use Optional.ofNullable for non-required fields.

For me the handling is absolutely correct. I see no reason to create empty optionals to act with null values.

That was also a conscious decision for the current implementation and Jackson is doing it correctly so far. If so, you only need this shortcut in your own code and there are other ways.

For this reason it should be a setting if at all.

However, this is incorrect why getting "present": true; the corrected value should be null if i didn't pass the value.

This is because you have set your Jackson Objectmapper incorrectly. You already need the Java 8 module to trade optionals. see https://www.baeldung.com/jackson-optional

Expected output

Your expected output depends on you Jackson setting. There is a setting what should happen when the field is absent: a) is visible with null b) is absent in the payload

Here you have all settings: https://github.com/FasterXML/jackson-databind/wiki/Mapper-Features and examples https://www.baeldung.com/spring-boot-customize-jackson-objectmapper

Kavan72 commented 6 months ago

If I don't specify that field as required in Swagger, the generator will create that field with Optional.of. At that point, if my value is null, it will throw a NullPointerException. In my opinion, we should use Optional.ofNullable for non-required fields.

For me the handling is absolutely correct. I see no reason to create empty optionals to act with null values.

That was also a conscious decision for the current implementation and Jackson is doing it correctly so far. If so, you only need this shortcut in your own code and there are other ways.

For this reason it should be a setting if at all.

However, this is incorrect why getting "present": true; the corrected value should be null if i didn't pass the value.

This is because you have set your Jackson Objectmapper incorrectly. You already need the Java 8 module to trade optionals. see https://www.baeldung.com/jackson-optional

Expected output

Your expected output depends on you Jackson setting. There is a setting what should happen when the field is absent: a) is visible with null b) is absent in the payload

Here you have all settings: https://github.com/FasterXML/jackson-databind/wiki/Mapper-Features and examples https://www.baeldung.com/spring-boot-customize-jackson-objectmapper

https://github.com/OpenAPITools/openapi-generator/issues/14765#issuecomment-1437450306

MelleD commented 6 months ago

https://github.com/OpenAPITools/openapi-generator/issues/14765#issuecomment-1570215549 https://github.com/OpenAPITools/openapi-generator/issues/14765#issuecomment-1948266832 https://github.com/OpenAPITools/openapi-generator/issues/14765#issuecomment-1948357891

And off course: https://github.com/OpenAPITools/openapi-generator/pull/17202 https://github.com/OpenAPITools/openapi-generator/pull/17202#discussion_r1407664380

And the whole discussion is of no use if your Jackson incorrectly serializes an Optional/JsonNullable. You have to set the settings correctly to have a good JSON

mvdwalle commented 1 week ago

As mentioned in the original description this gives e NPE when using a simple entity with an optional field. The generator creates a fluent api which is easy to use. However due to the use of Optional.of() in the generated methods of that fluent API a NPE is quickly introduced.

// Example snippet extracted from a real world example
class Pojo {
    private String fieldA;

    private Optional<String> fieldB = Optional.empty();

    private Optional<LocalDate> fieldC = Optional.empty();

    public Pojo fieldA(String fieldA) {
        this.fieldA = fieldA;
        return this;
    }

    public Pojo fieldB(String fieldB) {
        this.fieldB = Optional.of(fieldB); // If fieldB is null this generates an NPE
        return this;
    }

    public Pojo fieldC(LocalDate fieldC) {
        this.fieldC = Optional.of(fieldC); // If fieldC is null this generates an NPE
        return this; 
    }
}

var objectFromSomewhere = .... // Might come from the database or something else
var pojo = new Pojo // Pojo was generated from openapi spec with useOptional = true
    .fieldA(objectFromSomewhere.getFieldA())
    .fieldB(objectFromSomewhere.getFieldB()) // Possibly null and thus generates a NPE
    .fieldC(objectFromSomewhere.getFieldC()) // Possibly null
   // A few more fields here, some of them optional, some of them not.

This can quickly generate a NPE. This forces the consumer of the fluent API to do all kinds of null checks and defeating the purpose of having such a fluent API. A simple change of allowing null parameters for optional fields by changing the generated code to Optional.ofNullable() makes everyones live a lot easier.

It would be very nice to have this fixed @MelleD as it now forces us and all teams at my organisation to do all kinds of ugly checks.

Note! This has nothing todo with Jackson configuration, but with the fluent API becoming unusable

cc: @Kavan72

jwilmoth-ehs commented 1 week ago

Our team has run across this as well. Moving from Optional.of to Optional.ofNullable in the generated code would solve the issue for us.

Kavan72 commented 1 week ago

Our team has run across this as well. Moving from Optional.of to Optional.ofNullable in the generated code would solve the issue for us.

Does your team use a custom template to generate Optional.ofNullable inside Optional.of?

OpenAPITools / openapi-generator