Pactflow Bidirectional contract test fails in case of OneOf /AllOf in Open API spec file though the consumer pact is as per the spec file

[tl-madhulika-mitra]

Consumer: Cypress+MSW - Generated Pact file Provider - Open API Specification Using Pact broker

The response structure schema looks like this:

_GetStatusDetails{ oneOf ->	#/components/schemas/Authorization Required{...}#/components/schemas/Authorizing{...}#/components/schemas/Authorized{...}#/components/schemas/Executed{...}#/components/schemas/Settled{...}#/components/schemas/Failed{...}_

In the consumer pact file - we have added the required interaction for Authorization Required status.

On comparison all the fields error out and Response Body Is Incompatible Response body is incompatible with the response body schema in the spec file: should NOT have additional properties - id

Mismatched Pact Path: [root].interactions[0].response.body Mismatched Provider Path: [root].paths./status/{id}.get.responses.200.content.application/json; charset=UTF-8.schema.additionalProperties.

Similarly when our schemas had "AllOf" also , we encounter a failure. Once we remove these and make a simple response body which does not nest further, we are able to proceed.

Kindly let us know if there is an open issue for this or is there some specification we need to mention. I did not post this to slack, since after a lot of retries and check I do feel there is some issue here. But please correct me if im wrong :)

[YOU54F]

Hey @tl-madhulika-mitra, thanks for the message (and using the tool)

The validation is being performed by this tool - which is a fork of atlassians https://bitbucket.org/atlassian/swagger-mock-validator

A change in default behaviour was introduced

• see release note for the change https://github.com/pactflow/swagger-mock-validator/releases/tag/11.0.0

This thread is worth a read and it will provide some reasoning as to the issues in validation for allOf, anyOf and oneOf

https://bitbucket.org/atlassian/swagger-mock-validator/issues/84/test-incorrectly-passes-when-mock-expects

Using Pact broker

Just to confirm, you are using the Pactflow broker, not the OSS broker

[tl-madhulika-mitra]

Thank you @YOU54F . Yes we are using Pactflow broker.

• So to clarify - are we saying we should be adding "additional properties:false" in our swagger yaml files to go around the AllOf/OneOf issues or are they 2 different things?
• Also in here you mention - https://github.com/pactflow/swagger-mock-validator/releases/tag/11.0.0 - How could we do either of them? The flag can be set to true or you can drop down to v10.3.0
• I have been trying to set "additional properties:true" and that's getting down the failures but that does not look like is the ideal way, since then we are no longer having a strict validation and its like reverse of what was suggested in the threads
• "additional properties:true" is not really the default behaviour, since the moment I remove them or make it "additional properties:false" my tests fail
• Also I feel we should clearly add this in the documentation in pactflow, ignore me if it already is

[tl-madhulika-mitra]

@YOU54F - Also 2 follow up issues is

Without setting any additional properties

• when the consumer response does not have some fields(even mandatory ones) while spec defines them as mandatory the test passes

setting any additional properties:true

• When a field has a value which is outside the allowed list of enum values/or any failure w.r.t request body apart from status code - the failure occurs at that field and many other fields in the response, while they are right and as per the spec

[YOU54F]

If you could provide some code examples illustrating the points and the pact/specs in question, it would help tremendously in visualising.

We can collate the various examples to illustrate the points, you can see some in the readme on this page.

You would just need a sample pact file and swagger file, and you can show the output with the different combination of flags.

[YOU54F]

Hey @tl-madhulika-mitra

Docs updated here

https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#allof-support-and-other-logical-keywords

and this issue would be best tracked against this repo

https://github.com/pactflow/swagger-mock-validator/issues?q=is%3Aopen+is%3Aissue

where we would hope to be able to address this in the future

[tl-madhulika-mitra]

Thanks @YOU54F . Could you please clarify where do we add the "additionalPropertiesInResponse":true - when we are using pactflow, I tried to refer this , but not super clear where to add this setting.