v1.30.0 - Error Generating Tests for `oneOf` Data Type

[tillig]

I have a schema with a single operation POST /event where the body looks like:

{
  "event": {
    "startDateTime": "2024-02-07"
  }
}

In the portman-config.json I have an overwrite specified for the event.startDateTime value.

When using Portman 1.29.2, generating tests yields an error:

/Path/to/repro/node_modules/dot-object/index.js:118
        throw new Error("Trying to redefine non-empty obj['" + k + "']")
              ^

Error: Trying to redefine non-empty obj['startDateTime']
    at DotObject._fill (/Path/to/repro/node_modules/dot-object/index.js:118:15)
    at DotObject._fill (/Path/to/repro/node_modules/dot-object/index.js:114:10)
    at DotObject.str (/Path/to/repro/node_modules/dot-object/index.js:179:10)
    at setByPath (/Path/to/repro/node_modules/@apideck/portman/dist/utils/setByPath.js:41:16)
    at /Path/to/repro/node_modules/@apideck/portman/dist/application/overwrites/overwriteRequestBody.js:63:50
    at Array.map (<anonymous>)
    at overwriteRequestBodyJson (/Path/to/repro/node_modules/@apideck/portman/dist/application/overwrites/overwriteRequestBody.js:36:21)
    at overwriteRequestBody (/Path/to/repro/node_modules/@apideck/portman/dist/application/overwrites/overwriteRequestBody.js:12:46)
    at /Path/to/repro/node_modules/@apideck/portman/dist/application/overwrites/applyOverwrites.js:14:148
    at Array.map (<anonymous>)
    at applyOverwrites (/Path/to/repro/node_modules/@apideck/portman/dist/application/overwrites/applyOverwrites.js:6:25)
    at /Path/to/repro/node_modules/@apideck/portman/dist/application/TestSuite.js:228:40
    at Array.map (<anonymous>)
    at TestSuite.injectOverwrites (/Path/to/repro/node_modules/@apideck/portman/dist/application/TestSuite.js:225:22)
    at Portman.injectTestSuite (/Path/to/repro/node_modules/@apideck/portman/dist/Portman.js:337:19)
    at Portman.<anonymous> (/Path/to/repro/node_modules/@apideck/portman/dist/Portman.js:54:30)

Node.js v20.15.1

Under Portman 1.28.0 this works without error. Once it hits 1.29.0 I see the error.

I believe this has to do with oneOf on the property - the startDateTime may be a date (2024-02-07) or a date-time (2024-02-07T12:13:14Z) - in combination with using overwrites and the new empty value testing.

Click to see the JSON schema

```json { "components": { "schemas": { "dateTimeType": { "description": "Either a date-only or a date with a time.", "oneOf": [ { "format": "date-time" }, { "format": "date" } ], "type": "string" }, "event": { "description": "An object with a start date.", "properties": { "startDateTime": { "$ref": "#/components/schemas/dateTimeType" } }, "required": [ "startDateTime" ], "type": "object" }, "eventContainer": { "description": "Object that holds an event. The holder is important for the repro, if it's not here then the problem doesn't show.", "properties": { "event": { "$ref": "#/components/schemas/event" } }, "type": "object" } } }, "info": { "title": "Demo", "version": "v1" }, "openapi": "3.0.3", "paths": { "/event": { "post": { "description": "Creates an event.", "operationId": "post-event", "requestBody": { "content": { "application/json": { "schema": { "$ref": "#/components/schemas/eventContainer" } } } }, "responses": { "204": { "description": "Success." } }, "tags": [ "demo" ] } } }, "servers": [ { "description": "Demo API.", "url": "https://api.demo.com" } ], "tags": [ { "description": "Demo.", "name": "demo" } ] } ```

Click to see the Portman config JSON

```json { "overwrites": [ { "openApiOperationIds": [ "post-event" ], "overwriteRequestBody": [ { "key": "event.startDateTime", "overwrite": true, "value": "2002-12-01" } ] } ], "tests": { "contractTests": [ { "openApiOperation": "*::/*" } ] }, "version": 1 } ```

Click to see a simple package.json to drive the repro

Run `npm install && npm run build` to make it go. ```json { "devDependencies": { "@apideck/portman": "^1.30.0" }, "license": "ISC", "name": "specification", "scripts": { "build": "portman -l schema.json -o artifacts/tests/postman.json -c portman-config.json -b http://api", "clean": "rm -rf artifacts && mkdir -p artifacts", "prebuild": "npm run clean" }, "type": "commonjs", "version": "1.0.0" } ```

[thim81]

hi @tillig

Would this be the expected result?

I created a PR that would fix the issue, but i'm not sure about the result.

[tillig]

Yeah, I think that's right. This is actually a very, very pared down bit of a larger schema where the real replacement is a {{variable}} that has a value, but whether it's a variable or a plain value, the exception message ends up being the same.

[thim81]

My assumption is that with oneOf or anyOf, the overwrite should take the 1st item since there could be multiple, and use that object to apply the overwrite. The current implementation overwrites the whole object.

I will dive in deeper.

[thim81]

hi @tillig

After some investigating, I have found the actual cause. The OpenAPI was missing a type per oneOf (with assistance from ChatGPT).

If we modified the spec from

components:
  schemas:
    dateTimeType:
      description: Either a date-only or a date with a time.
      oneOf:
        - format: date-time
        - format: date
      type: string

to

components:
  schemas:
    dateTimeType:
      description: Either a date-only or a date with a time.
      oneOf:
        - type: string
          format: date-time
        - type: string
          format: date

The conversion (v1.30.1) did properly process it (without error) resulting in this:

[tillig]

While I don't mind changing the schema, based on examples like this one from json-schema.org...

{
  "type": "number",
  "oneOf": [
    { "multipleOf": 5 },
    { "multipleOf": 3 }
  ]
}

...it seems like having the type: string outside the oneOf should be legal.

[thim81]

hi @tillig

We have just released Portman v1.30.2, this contains the improvements to prevent the error with the oneOf schema.

If this works for you, feel free to close this issue.

[tillig]

I'm still working on getting a chance to test this. I haven't forgotten, I just haven't been at a keyboard.

[thim81]

@tillig No worries, there is no time pressure. Being away from the keyboard is usually better, since it could mean other fun activities.

[tillig]

Verified - 1.30.2 works great. Thanks!