OpenAPITools / openapi-generator

OpenAPI Generator allows generation of API client libraries (SDK generation), server stubs, documentation and configuration automatically given an OpenAPI Spec (v2, v3)
https://openapi-generator.tech
Apache License 2.0
20.73k stars 6.32k forks source link

[BUG] [JAVA] openapi-generator-maven-plugin does not handle allOf composed of multiple objects #10010

Open norbjd opened 2 years ago

norbjd commented 2 years ago

Bug Report Checklist

Description

Using allOf composed of multiple objects in a response does not generate a class combining these multiple objects. Instead it uses the first class defined in the allOf section.

openapi-generator version

Version 5.2.0:

<plugin>
    <groupId>org.openapitools</groupId>
    <artifactId>openapi-generator-maven-plugin</artifactId>
    <version>5.2.0</version>
</plugin>

I have also tested with master (5.2.1-SNAPSHOT), commit fe9636e9.

OpenAPI declaration file content or url

Important part here is the allOf:

openapi: 3.0.0
info:
  title: Test
  version: v1
paths:
  /test:
    get:
      operationId: get
      responses:
        '200':
          description: Get
          content:
            application/json:
              schema:
                allOf:
                  - $ref: '#/components/schemas/A'
                  - $ref: '#/components/schemas/B'
components:
  schemas:
    A:
      type: object
      properties:
        attA:
          type: string
    B:
      type: object
      properties:
        attB:
          type: integer
Generation Details
generatorName: java
inputSpec: openapi.yaml
Steps to reproduce
 mvn org.openapitools:openapi-generator-maven-plugin:5.2.0:generate \
    -Dopenapi.generator.maven.plugin.inputSpec=openapi.yaml \
    -Dopenapi.generator.maven.plugin.generatorName=java

Shows a warning:

[WARNING] allOf with multiple schemas defined. Using only the first one: A

It generates classes A and B. But when calling get(), the value returned by the call is A:

DefaultApi api = new DefaultApi();
A a = api.get();

I expected instead a composite object with A and B properties (attA and attB), like this (result from https://editor.swagger.io/):

image

Related issues/PRs
Suggest a fix
zczhuohuo commented 2 years ago

any update on this?

paulbors commented 2 years ago

The bug is in the code-generator falling back to a raw Object class when both a schema ref and inline is used such as:

...
      responses:
        '200':
          description: Some description
          content:
            application/json:
              schema:
                title: MyCustomClassNameResponse
                allOf:
                  - type: object
                    properties:
                      data:
                        $ref: '#/components/schemas/Dog'
                  - $ref: '#/components/schemas/Pet'
...

If you comment out the first inline object, you end up with just Pet, and if you comment the second reference you end up just with Dog.

As work-around you can edit the spec and introduce your own return type and use a single ref there similar to:

...
      responses:
        '200':
          description: Some description
          content:
            application/json:
              schema:
                  $ref: '#/components/schemas/MyCustomClassNameResponse'
...

Not don't claim you can't edit the spec, simply download it, modify it, check it in your source code and generate from it !!!

norbjd commented 2 years ago

@paulbors of course editing the spec is possible (and that's the solution we used now), but this is not very convenient if the openapi evolves : we have to manually change our version to match the new version.

Besides, as you said editing the openapi is just a workaround.

That's why I opened this issue : the fact that the generator does not generate right classes when providing a valid openapi spec (as in my example) is probably a bug and as so probably needs to be fixed.

paulbors commented 2 years ago

I was referring to your comments here: https://stackoverflow.com/questions/68773761/openapi-generator-maven-plugin-java-does-not-handle-allof-properly

Right, so we know is not working with:

We know it works with:

So that's the real bug here. Now let's see which one of us (you or me) needs this to work first and puts out a Pull Request to this repo fixing it.

The beauty of open source code... :)

Richardmbs12 commented 2 years ago

For me a "A single $ref" also doesnt work when using allOf :(

However my case is a bit different:

Customer:
  allOf:
    - $ref: '#/components/schemas/CreateCustomer'
  required:
    - customerId
  properties:
    customerId:
      type: string

I get a java class that only has "customerId" field

maddin79 commented 2 years ago

Hello everyone,

any progress on this. We have a lot of combined schema and need this functionality. I'm rahter new to OpenAPI, so @paulbors could you please clarify your workaround. I do not get what I have to do. Would appreciate it.

If someone could point me to the specific code where the problem is, I could have a look to fix it.

nicobo commented 2 years ago

I guess this is the cause for other issues :

And I have the same issue with html2 generators also...

KSDaemon commented 2 years ago

Also ran into this bug :( In my case i have a spec with ~15k LOC with compositions of cross linked schemas. So editing every allOf to replace it with single inline scheme is not a right way %(

cdprete commented 2 years ago

I have the same issue also when generating plain models...

skwasniak commented 2 years ago

In my case, initially I wrote:

  User:
    allOf:
      - $ref: '#/definitions/UserBase'
      - type: "object"
        properties:
          password:
          type: "string"
          format: password

Where the correct is: What worked for me is:

  User:
    allOf:
      - $ref: '#/definitions/UserBase'
    type: "object"
    properties:
      password:
        type: "string"
        format: password

After applying the change warning is gone.

I'm using kotlin-spring generator and swagger: "2.0".

argenstijn commented 2 years ago

The warning is gone butt still it does not work (using java)

argenstijn commented 2 years ago

Can this please be fixed!!

skwasniak commented 2 years ago

The warning is gone butt still it does not work (using java)

It works in my case i.e. kotlin and kotlin-spring generators.

$ openapi-generator-cli version
5.4.0

Maybe providing a code snippets would be helpful for fixing it in this case.

koalo commented 2 years ago

@skwasniak I am not sure if your second option is really correct. On https://swagger.io/docs/specification/data-models/oneof-anyof-allof-not/ you can find the following example:

       allOf: # Combines the main `Pet` schema with `Cat`-specific properties 
        - $ref: '#/components/schemas/Pet'
        - type: object
          # all other properties specific to a `Cat`
          properties:
            hunts:
              type: boolean
            age:
              type: integer

What am I getting wrong here?

skwasniak commented 2 years ago

@koalo I'm not saying this is accordingly to open api spec, I'm just stating a workaround that works for kotlin and kotlin-spring generators. But you have the point, my wording "Where the correct is:" is misleading.

Anyhow it is a bug that needs attention, as probably it is widely used feature.

P.S. My API spec is swagger: "2.0".

consultantleon commented 2 years ago

Upgrading from swagger v2 to swagger v3 we hit exactly this issue when generating java code.

In swagger v2 codegen, allOf types would nicely be composed.

In swagger v3 only the first type in the allOf would become part of the model...

As the OpenAPITool community seems more active I now upgraded to OpenAPITools codegen.

Although it still logs a nonsense warning about multiple types in allOf (please merge https://github.com/OpenAPITools/openapi-generator/pull/7446 or other PR removing this warning in the correct way!):

[main] WARN o.o.codegen.DefaultCodegen - allOf with multiple schemas defined. Using only the first one: XYZ [main] WARN o.o.codegen.DefaultCodegen - More than one inline schema specified in allOf:. Only the first one is recognized. All others are ignored.

as a matter of fact the generator 5.4.0 does correctly generate the complete composed java model!

nikosmoum commented 2 years ago

Using 5.4.0 with Java and agree with @consultantleon. The generated model is composed correctly from all schemas defined under the allOf, but this warning message is clearly wrong (allOf is supposed to be used with multiple schemas).

skwasniak commented 2 years ago

I've upgraded to 3.0.3 API version with a help of the generator (openapi-generator-cli generate -i <input_file> -g openapi-yaml) and the result is the following:

    User:
      allOf:
        - $ref: '#/components/schemas/UserBase'
      properties:
        password:
          type: string
          format: password

Generation of kotlin works correctly. openapi-generator-cli version is 5.4.0.

fastluca commented 2 years ago

Also in case of <generatorName>spring</generatorName> there is the same warning and only the first ref is used in the generation of the model. Tested with plugin version 5.4.0 and 6.0.0-beta

norbjd commented 2 years ago

@consultantleon @nikosmoum @skwasniak I am not sure how you got a complete composite object. I've tried with the example in my first post, using the latest versions at the time of the plugin (5.4.0 and 6.0.0-beta) and the latest supported openapi spec (3.0.3), and I still got the same result. Maybe you have a different case than the example in my first post.

In my case, the Java DefaultApi class generated looks like this:

public class DefaultApi {
    // ...

    public A get() throws ApiException {
        ApiResponse<A> localVarResp = getWithHttpInfo();
        return localVarResp.getData();
    }

    // ...
}

The get() method does not return a composite object of A and B, but just A.

Also, I've tried with Kotlin:

 mvn org.openapitools:openapi-generator-maven-plugin:5.4.0:generate \
    -Dopenapi.generator.maven.plugin.inputSpec=openapi.yaml \
    -Dopenapi.generator.maven.plugin.generatorName=kotlin

And got the same result (the method get() only return A:

class DefaultApi(basePath: kotlin.String = defaultBasePath) : ApiClient(basePath) {
    // ...

    fun get() : A {
        val localVarResponse = getWithHttpInfo()

        return when (localVarResponse.responseType) {
            // ...
        }
    }

    // ...
}

As @fastluca and others said, the bug is still present.

By the way, these 3 OpenAPI specs parts (replace in my initial openapi.yaml):

schema:
  allOf:
    - $ref: '#/components/schemas/A'
  properties:
    b:
      type: integer

and:

schema:
  allOf:
    - $ref: '#/components/schemas/A'
    - type: object
      properties:
        b:
          type: integer

and:

schema:
  allOf:
    - $ref: '#/components/schemas/A'
  type: "object"
  properties:
    b:
      type: integer

give exactly the same result in Java and Kotlin (get() method always returns A, and the b field does not appear anywhere).

leonard84 commented 1 year ago

What I can observe with id("org.openapi.generator") version "6.0.1", is that if you define a schema

components:
  schemas:
    CustomQuery:
      allOf:
      - type: object
        required:
        - container
        properties:
          container:
            type: string
            default: '*'
      - $ref: '#/components/schemas/CommonQuery'

Then it will generate two classes, one CustomQuery which has all the fields from the CommonQuery and the locally defined ones, and CustomQueryAllOf which only has the container field.

The same happens if you use allOf directly for a query parameter with will generate a class, like GetMyQueryParameter, and GetMyQueryParameterAllOf, where the first one is correct and the latter one only has the one field. You get the allOf with multiple schemas defined. Using only the first one: ... warning for both.

However, the *AllOf classes were actually unused in the code, but they are wrongly imported instead of the correct type without the AllOf ending, leading to compile errors.

Update: explicitly setting x-all-of-name: CustomQuery seems to fix import the problem.

Thul95 commented 1 year ago

Any update? This is still a blocker in 6.1.0

edigeek commented 1 year ago

Any update on this? I am using 6.2.0 and still getting this warning and compilation errors in models generated.

JayPeet commented 1 year ago

Also experencing this issue with the C# dotnet Core generator.

temofey1989 commented 1 year ago

+1 Same issue...

nikimicallef commented 1 year ago

+1 Same issue

carloslmeIndi commented 1 year ago

+1 same issue

on-delete commented 1 year ago

+1 same issue

skafendre commented 1 year ago

+1, issue appear while using the html2 generator

KaiserP commented 1 year ago

+1 same issue

davidlinner commented 1 year ago

+1 same issue

Morakir commented 1 year ago

+1 same issue

vafokina commented 1 year ago

+1 same issue

Serob commented 1 year ago

+1 same issue

nurimanovo commented 1 year ago

+1 same issue

argenstijn commented 1 year ago

+1 same issue

kirimati commented 1 year ago

+1 same issue

guillaume-thomas commented 1 year ago

+1 same issue

wing328 commented 1 year ago

I tested with the spec provided in https://github.com/OpenAPITools/openapi-generator/issues/10010#issue-950490570 and cannot reproduce the issue and the output is what the users want as follows:

  public static final String SERIALIZED_NAME_ATT_A = "attA";
  @SerializedName(SERIALIZED_NAME_ATT_A)
  private String attA;

  public static final String SERIALIZED_NAME_ATT_B = "attB";
  @SerializedName(SERIALIZED_NAME_ATT_B)
  private Integer attB;

can someone please share another spec to more easily reproduce the issue?

norbjd commented 1 year ago

@wing328 thanks for your help, it looks like it works now since 6.0.0. The command:

mvn org.openapitools:openapi-generator-maven-plugin:6.0.0:generate \
    -Dopenapi.generator.maven.plugin.inputSpec=openapi.yaml \
    -Dopenapi.generator.maven.plugin.generatorName=java

creates a new class Get200Response here: target/generated-sources/openapi/src/main/java/org/openapitools/client/model/Get200Response.java. This class was not generated before version 6.0.0 (there were only A and B classes).

This Get200Response contains both attributes (as in your last message), and the get method of DefaultApi returns a Get200Response:

public Get200Response get() throws ApiException {
    ApiResponse<Get200Response> localVarResp = getWithHttpInfo();
    return localVarResp.getData();
}

So this should work for my case (I'm not able to test right now, but this should be ok):

DefaultApi api = new DefaultApi();
Get200Response r = api.get();
System.out.println(r.getAttA());
System.out.println(r.getAttB());
fbett commented 1 year ago

The problem still exists in version 6.4.0.

Example: https://developers.pipedrive.com/docs/api/v1/openapi.yaml

derkd commented 1 year ago

@wing328 try to test it with this code yaml

   A:
      type: object
      properties:
        attA:
          type: string
    B:
      allOf:
        - "$ref": "#/components/schemas/A"
      type: object
      properties:
        attB:
          type: integer

    C:
      allOf:
        - "$ref": "#/components/schemas/B"

I would expect a C class with AttA and AttB but it only generates it with AttA

I use the latest 6.4.0

wing328 commented 1 year ago

thanks for sharing the spec to reproduce the issue.

if anyone would like to sponsor the fix to make it a higher priority, please reply to let us know.

fbett commented 1 year ago

Thanks @wing328 for your efforts. Great project by the way!

I have sent a small bounty of 50$

NiklasLoechte commented 1 year ago

I've found something that might interest you. By adding a vendor flag "x-parent": "abstract" you can make it work (at least in my case, spring-java and openapi-generator v6.4.0).

The Schema I found under: modules/openapi-generator/src/test/resources/3_0/allOf_extension_parent.yaml looks like this:

components:
  schemas:
    Person:
      description: person using x-parent (abstract) to indicate it's a parent class
      type: object
      x-parent: "abstract"
      properties:
        $_type:
          type: string
        lastName:
          type: string
        firstName:
          type: string
    Child:
      description: A representation of a child
      allOf:
      - type: object
        properties:
          age:
            type: integer
            format: int32
      - $ref: '#/components/schemas/Person'
Zajs commented 1 year ago

We have the same issue with the specification from coda.io https://coda.io/apis/v1/openapi.yaml

wing328 commented 1 year ago

@derkd I've filed https://github.com/OpenAPITools/openapi-generator/pull/15039 to add a new openapi-generator normalizer rule to handle properties and allOf in the same level.

Tested with your spec and the output compiles without issue.

     getUserCallLogsResponse200:
       allOf:
       - $ref: '#/components/schemas/baseResponse'
-      example:
-        additional_data:
-          start: 0
-          limit: 6
-          more_items_in_collection: true
-        data:
-        - null
-        - null
-      properties:
-        data:
-          items:
-            $ref: '#/components/schemas/responseCallLogObject'
-          type: array
-        additional_data:
-          $ref: '#/components/schemas/fieldsResponse200_allOf_additional_data'
+      - $ref: '#/components/schemas/getUserCallLogsResponse200_allOf'
       title: getUserCallLogsResponse200

Command:

java -jar modules/openapi-generator-cli/target/openapi-generator-cli.jar generate -g java -i https://developers.pipedrive.com/docs/api/v1/openapi.yaml -o /tmp/allofbugfix3/ --openapi-normalizer REFACTOR_ALLOF_WITH_PROPERTIES_ONLY=true,SIMPLIFY_BOOLEAN_ENUM=true
wing328 commented 1 year ago

@fbett thanks for the sponsorship 👍

wing328 commented 1 year ago

PR merged. Please give it a try with the latest master via SNAPSHOT jar, docker images or build the project locally.

TomasTokaMrazek commented 1 year ago

@wing328 I am not sure, if it's actually fixed or if my plugin configuration is wrong (really lacking documentation and examples here) or if it does actually work and WARN log messages are not relevant.

Specs: WSO2 APIM 4.0.0 - Admin V2 Plugin configuration:

            <plugin>
                <groupId>org.openapitools</groupId>
                <artifactId>openapi-generator-maven-plugin</artifactId>
                <version>6.5.0</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>generate</goal>
                        </goals>
                        <configuration>
                            <inputSpec>${project.basedir}/src/main/resources/4.0.0/admin-v2.yaml</inputSpec>
                            <generatorName>java</generatorName>
                            <library>native</library>
                            <output>${project.basedir}</output>
                            <modelPackage>com.example.client.v400.admin.model</modelPackage>
                            <generateSupportingFiles>false</generateSupportingFiles>
                            <generateModelDocumentation>false</generateModelDocumentation>
                            <generateModelTests>false</generateModelTests>
                            <generateApis>false</generateApis>
                            <openapiNormalizer>REFACTOR_ALLOF_WITH_PROPERTIES_ONLY=true</openapiNormalizer>
                            <configOptions>
                                <useJakartaEe>true</useJakartaEe>
                                <hideGenerationTimestamp>true</hideGenerationTimestamp>
                            </configOptions>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

Log entries:

[WARNING] allOf with multiple schemas defined. Using only the first one: ThrottlePolicy
[WARNING] allOf with multiple schemas defined. Using only the first one: ThrottlePolicy
[WARNING] allOf with multiple schemas defined. Using only the first one: ThrottlePolicy
[WARNING] allOf with multiple schemas defined. Using only the first one: ThrottlePolicy
[WARNING] More than one inline schema specified in allOf:. Only the first one is recognized. All others are ignored.
[WARNING] allOf with multiple schemas defined. Using only the first one: ThrottlePolicy
[WARNING] allOf with multiple schemas defined. Using only the first one: ThrottleLimitBase
[WARNING] allOf with multiple schemas defined. Using only the first one: ThrottleLimitBase
[WARNING] allOf with multiple schemas defined. Using only the first one: ThrottleLimitBase

I need model only, as I plan to use Spring 6 declarative HTTP interface services with webclient proxy. AFAIK this library is not implemented in the generator.