apideck-libraries / portman

Port OpenAPI Specs to Postman Collections, inject test suite and run via Newman 👨🏽‍🚀
http://getportman.com/
Apache License 2.0
638 stars 59 forks source link

Reused schemas only use default data type value in first occurance #571

Closed mhpkmd closed 5 months ago

mhpkmd commented 7 months ago

It seems like when your openapi spec is reusing the same schema in multiple endpoints (or when referring to the same schema from other schemas), portman does only generate default data type values on the first occurrence (i.e. a integer type -65321). On all following occurrences the field of same type seems to be treated as a string (i.e. same integer type would be "" as raw body. This happens on all properties where example field is not defined.

Very simple openapi taken from Swaggers Pet shop examples and scaled down:

INPUT { "openapi": "3.0.3", "info": { "title": "Swagger Petstore - OpenAPI 3.0", "description": "This is a sample Pet Store Server based on the OpenAPI 3.0 specification. You can find out more about\nSwagger at [https://swagger.io](https://swagger.io). In the third iteration of the pet store, we've switched to the design first approach!\nYou can now help us improve the API whether it's by making changes to the definition itself or to the code.\nThat way, with time, we can improve the API in general, and expose some of the new features in OAS3.\n\nSome useful links:\n- [The Pet Store repository](https://github.com/swagger-api/swagger-petstore)\n- [The source API definition for the Pet Store](https://github.com/swagger-api/swagger-petstore/blob/master/src/main/resources/openapi.yaml)", "termsOfService": "http://swagger.io/terms/", "contact": { "email": "apiteam@swagger.io" }, "license": { "name": "Apache 2.0", "url": "http://www.apache.org/licenses/LICENSE-2.0.html" }, "version": "1.0.11" }, "externalDocs": { "description": "Find out more about Swagger", "url": "http://swagger.io" }, "servers": [ { "url": "https://petstore3.swagger.io/api/v3" } ], "security": [ ], "tags": [ { "name": "user", "description": "Operations about user" } ], "paths": { "/user": { "post": { "tags": [ "user" ], "summary": "Create user", "description": "This can only be done by the logged in user.", "operationId": "createUser", "requestBody": { "description": "Created user object", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/User" } } } }, "responses": { "202": { "description": "202 Accepted." } } }, "put": { "tags": [ "user" ], "summary": "Update user", "description": "This can only be done by the logged in user.", "operationId": "updateUser", "requestBody": { "description": "Update user object", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/User" } } } }, "responses": { "200": { "description": "200 OK." } } } } }, "components": { "schemas": { "User": { "title": "UserDTO", "description": "", "type": "object", "additionalProperties": false, "nullable": false, "properties": { "id": { "type": "integer", "format": "int64", "example": 10 }, "id-no-example": { "type": "integer", "format": "int64" }, "uuid":{ "type": "string", "format": "uuid", "example": "5f2c7c30-3ed9-43bb-b77e-15e879e160db" }, "uuid-no-example":{ "type": "string", "format": "uuid" }, "uuids":{ "type": "array", "items":{ "type": "string", "format": "uuid" }, "example": ["4ce807d0-d4c5-4ab5-b318-fb430f658ad1"] }, "uuids-no-example":{ "type": "array", "items":{ "type": "string", "format": "uuid" } }, "string": { "type": "string", "example": "theUser" }, "string-no-example": { "type": "string" }, "boolean": { "type": "boolean", "example": true }, "boolean-no-example": { "type": "boolean" }, "int32": { "type": "integer", "format": "int32", "example": 1 }, "int32-no-example": { "type": "integer", "format": "int32" }, "date-time": { "type": "string", "format": "date-time", "example": "2024-03-20T10:11:54Z" }, "date-time-no-example": { "type": "string", "format": "date-time" }, "number": { "type": "number", "format": "decimal", "example": 43.25 }, "number-no-example": { "type": "number", "format": "decimal" } } } }, "securitySchemes": { "petstore_auth": { "type": "oauth2", "flows": { "implicit": { "authorizationUrl": "https://petstore3.swagger.io/oauth/authorize", "scopes": { "write:pets": "modify pets in your account", "read:pets": "read your pets" } } } }, "api_key": { "type": "apiKey", "name": "api_key", "in": "header" } } } }

OUTPUT { "_": { "postman_id": "145d1020-ccca-4e71-8025-1953fd34b034" }, "item": [ { "id": "759a8afa-bfa8-42af-bb2e-c13310d4fc8d", "name": "user", "description": { "content": "Operations about user", "type": "text/plain" }, "item": [ { "id": "2d0ead94-35be-4b94-9a68-6e22da5854cd", "name": "Create user", "request": { "name": "Create user", "description": { "content": "This can only be done by the logged in user.", "type": "text/plain" }, "url": { "path": [ "user" ], "host": [ "{{baseUrl}}" ], "query": [], "variable": [] }, "header": [ { "key": "Content-Type", "value": "application/json" } ], "method": "POST", "body": { "mode": "raw", "raw": "{\n \"id\": 10,\n \"id-no-example\": 10861459,\n \"uuid\": \"5f2c7c30-3ed9-43bb-b77e-15e879e160db\",\n \"uuid-no-example\": \"urn:uuid:92e9a1e2-5605-fd8f-9a4b-ff59ed5f3ff0\",\n \"uuids\": [\n \"4ce807d0-d4c5-4ab5-b318-fb430f658ad1\"\n ],\n \"uuids-no-example\": [\n \"15a6e80e-28d2-e92b-ca7a-82468b4cd4c2\",\n \"f201bed8-1e3b-933b-cb41-42659d07a225\"\n ],\n \"string\": \"theUser\",\n \"string-no-example\": \"aliqua ea\",\n \"boolean\": true,\n \"boolean-no-example\": false,\n \"int32\": 1,\n \"int32-no-example\": -74363662,\n \"date-time\": \"2024-03-20T10:11:54Z\",\n \"date-time-no-example\": \"1988-09-22T20:16:04.722Z\",\n \"number\": 43.25,\n \"number-no-example\": 77460913.5383009\n}", "options": { "raw": { "language": "json" } } } }, "response": [ { "_": { "postman_previewlanguage": "text" }, "id": "d8e23ce0-9c07-4fc4-85eb-eff18e57162f", "name": "202 Accepted.", "originalRequest": { "url": { "path": [ "user" ], "host": [ "{{baseUrl}}" ], "query": [], "variable": [] }, "method": "POST", "body": { "mode": "raw", "raw": "{\n \"id\": \"<long>\",\n \"id-no-example\": \"<long>\",\n \"uuid\": \"<uuid>\",\n \"uuid-no-example\": \"<uuid>\",\n \"uuids\": [\n \"<uuid>\",\n \"<uuid>\"\n ],\n \"uuids-no-example\": [\n \"<uuid>\",\n \"<uuid>\"\n ],\n \"string\": \"<string>\",\n \"string-no-example\": \"<string>\",\n \"boolean\": \"<boolean>\",\n \"boolean-no-example\": \"<boolean>\",\n \"int32\": \"<integer>\",\n \"int32-no-example\": \"<integer>\",\n \"date-time\": \"<dateTime>\",\n \"date-time-no-example\": \"<dateTime>\",\n \"number\": \"<decimal>\",\n \"number-no-example\": \"<decimal>\"\n}", "options": { "raw": { "language": "json" } } } }, "status": "Accepted", "code": 202, "header": [ { "key": "Content-Type", "value": "text/plain" } ], "body": "", "cookie": [] } ], "event": [ { "listen": "test", "script": { "id": "87e196dc-ec47-4558-98d8-892f96edc126", "type": "text/javascript", "exec": [ "// Validate status 2xx \npm.test(\"[POST]::/user - Status code is 2xx\", function () {\n pm.response.to.be.success;\n});\n" ] } } ], "protocolProfileBehavior": { "disableBodyPruning": true } }, { "id": "b9c44075-3b52-4670-9629-9902c094a91f", "name": "Update user", "request": { "name": "Update user", "description": { "content": "This can only be done by the logged in user.", "type": "text/plain" }, "url": { "path": [ "user" ], "host": [ "{{baseUrl}}" ], "query": [], "variable": [] }, "header": [ { "key": "Content-Type", "value": "application/json" } ], "method": "PUT", "body": { "mode": "raw", "raw": "{\n \"id\": 10,\n \"id-no-example\": \"<long>\",\n \"uuid\": \"5f2c7c30-3ed9-43bb-b77e-15e879e160db\",\n \"uuid-no-example\": \"<uuid>\",\n \"uuids\": [\n \"4ce807d0-d4c5-4ab5-b318-fb430f658ad1\"\n ],\n \"uuids-no-example\": [\n \"<uuid>\",\n \"<uuid>\"\n ],\n \"string\": \"theUser\",\n \"string-no-example\": \"<string>\",\n \"boolean\": true,\n \"boolean-no-example\": \"<boolean>\",\n \"int32\": 1,\n \"int32-no-example\": \"<integer>\",\n \"date-time\": \"2024-03-20T10:11:54Z\",\n \"date-time-no-example\": \"<dateTime>\",\n \"number\": 43.25,\n \"number-no-example\": \"<decimal>\"\n}", "options": { "raw": { "language": "json" } } } }, "response": [ { "_": { "postman_previewlanguage": "text" }, "id": "3fc70ee8-2295-46fd-9336-2c5f04bc313a", "name": "200 OK.", "originalRequest": { "url": { "path": [ "user" ], "host": [ "{{baseUrl}}" ], "query": [], "variable": [] }, "method": "PUT", "body": { "mode": "raw", "raw": "{\n \"id\": \"<long>\",\n \"id-no-example\": \"<long>\",\n \"uuid\": \"<uuid>\",\n \"uuid-no-example\": \"<uuid>\",\n \"uuids\": [\n \"<uuid>\",\n \"<uuid>\"\n ],\n \"uuids-no-example\": [\n \"<uuid>\",\n \"<uuid>\"\n ],\n \"string\": \"<string>\",\n \"string-no-example\": \"<string>\",\n \"boolean\": \"<boolean>\",\n \"boolean-no-example\": \"<boolean>\",\n \"int32\": \"<integer>\",\n \"int32-no-example\": \"<integer>\",\n \"date-time\": \"<dateTime>\",\n \"date-time-no-example\": \"<dateTime>\",\n \"number\": \"<decimal>\",\n \"number-no-example\": \"<decimal>\"\n}", "options": { "raw": { "language": "json" } } } }, "status": "OK", "code": 200, "header": [ { "key": "Content-Type", "value": "text/plain" } ], "body": "", "cookie": [] } ], "event": [ { "listen": "test", "script": { "id": "d65627a6-9d19-45f3-b374-2da2a5f3c5fb", "type": "text/javascript", "exec": [ "// Validate status 2xx \npm.test(\"[PUT]::/user - Status code is 2xx\", function () {\n pm.response.to.be.success;\n});\n" ] } } ], "protocolProfileBehavior": { "disableBodyPruning": true } } ], "event": [] } ], "event": [], "variable": [ { "type": "string", "value": "https://petstore3.swagger.io/api/v3", "key": "baseUrl" } ], "info": { "_postman_id": "145d1020-ccca-4e71-8025-1953fd34b034", "name": "Swagger Petstore - OpenAPI 3.0", "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json", "description": { "content": "This is a sample Pet Store Server based on the OpenAPI 3.0 specification. You can find out more about\nSwagger at [https://swagger.io](https://swagger.io). In the third iteration of the pet store, we've switched to the design first approach!\nYou can now help us improve the API whether it's by making changes to the definition itself or to the code.\nThat way, with time, we can improve the API in general, and expose some of the new features in OAS3.\n\nSome useful links:\n- [The Pet Store repository](https://github.com/swagger-api/swagger-petstore)\n- [The source API definition for the Pet Store](https://github.com/swagger-api/swagger-petstore/blob/master/src/main/resources/openapi.yaml)\n\nContact Support:\n Email: apiteam@swagger.io", "type": "text/plain" } } }

Imported collections in POSTMAN looks like this:

First occurrence image

Second occurrence image

Expected result I was expected that portman would generate default data type values on all properties where examples are not defined, on every occurrence of a schema.

thim81 commented 7 months ago

hi @mhpkmd

That looks like unwanted behavior. We will have to run some validations to see where this behavior is originating.

thim81 commented 7 months ago

hi @mhpkmd

We just converted the provided OpenAPI file without any special configuration and it was always using the provided examples:

IMG-2024-04-08 at 16-48-04

IMG-2024-04-08 at 16-49-11

Could it be that you are using an older version of Portman?

mhpkmd commented 6 months ago

Hi @thim81

The issue persist in my end.

Took the same data as in the INPUT field of the original post, inserted it into a new file (openapi3_0-input.json). Ran the command below and the issue with types in the PUT endpoint persist.

Command portman init -l .\openapi\openapi3_0-input.json -o .\output\openapi3_0-output.postman_collection.example.json -s .\config\postman-config.example.default.json

Running portman v1.26.5 (portman --verison) which seems to be the newest

Also, please notice that the issue reported in visible in the output text even before importing it into Postman like highlighted in the picture below: image

thim81 commented 6 months ago

hi @mhpkmd

Your command uses a custom Postman config (postman-config.example.default.json) portman init -l .\openapi\openapi3_0-input.json -o .\output\openapi3_0-output.postman_collection.example.json -s .\config\postman-config.example.default.json

What is the content of the postman-config.example.default.json?

Can you try to do the conversion without the -s .\config\postman-config.example.default.json?

There is a Postman configuration parameter exampleParametersResolution that influences how the request body is rendered: using the example values OR the schema (aka integer, string, ...).

mhpkmd commented 6 months ago

hi @mhpkmd

Your command uses a custom Postman config (postman-config.example.default.json) portman init -l .\openapi\openapi3_0-input.json -o .\output\openapi3_0-output.postman_collection.example.json -s .\config\postman-config.example.default.json

What is the content of the postman-config.example.default.json?

Can you try to do the conversion without the -s .\config\postman-config.example.default.json?

There is a Postman configuration parameter exampleParametersResolution that influences how the request body is rendered: using the example values OR the schema (aka integer, string, ...).

@thim81 The output is the same when I run the following command (without option): portman init -l .\openapi\openapi3_0-input.json -o .\output\openapi3_0-output.postman_collection.example.json

The option file looks like this: { "folderStrategy": "Tags", "requestParametersResolution": "Example", "exampleParametersResolution": "Schema", "keepImplicitHeaders": true, "enableOptionalParameters": false }

Actually I just realized that when I don't set option explicit, portman is using default (portman-config.default.json). No such file exist in my root directory so I am a little in doubt of what options file it is actually using in this case. image

thim81 commented 6 months ago

hi @mhpkmd

Can you try to modify your Postman config file to use "exampleParametersResolution": "Example":

{
  "folderStrategy": "Tags",
  "requestParametersResolution": "Example",
  "exampleParametersResolution": "Example",
  "keepImplicitHeaders": true,
  "enableOptionalParameters": false
}

This will influence which values will be used when creating the request in Postman (details):

Select whether to generate the response parameters based on the schema or the example in the schema.

mhpkmd commented 6 months ago

hi @thim81

Forget my previous comment (now deleted). It actually seems to work with these settings. I'll run some tests in a larger scale to either confirm or deny if this actually fixed it.

thim81 commented 6 months ago

After your testing, if it is working as expected, feel free to close the issue. If you run into issues, let us know.

mhpkmd commented 5 months ago

I havn't been able to reproduce the issue after these settings, even though I was sure that I already tried that setup. Closing the issue, and thanks for the clarification.