pact-foundation / pact-jvm

JVM version of Pact. Enables consumer driven contract testing, providing a mock service and DSL for the consumer project, and interaction playback and verification for the service provider project.
https://docs.pact.io
Apache License 2.0
1.08k stars 479 forks source link

Allow array matchers to work when root of the body is an array (JavaScript) #562

Open benilovp opened 6 years ago

benilovp commented 6 years ago

I am running into issues using the eachLike matcher is PactJs. My provider returns a JSON in the following format:

[
    {
        "documentId": 0,
        "documentCategoryId": 5,
        "documentCategoryCode": null,
        "contentLength": 0,
        "tags": null,
    },
    {
        "documentId": 1,
        "documentCategoryId": 5,
        "documentCategoryCode": null,
        "contentLength": 0,
        "tags": null,
    }...
]

I have written a matcher for the mandatory fields. As you can see there are some nullable fields here which I do not want to match.

const EXPECTED_BODY = Pact.Matchers.eachLike({
    "name": Pact.Matchers.somethingLike("Test"),
    "documentId": Pact.Matchers.somethingLike(0),
    "documentCategoryId": Pact.Matchers.somethingLike(5),
    "contentLength": Pact.Matchers.somethingLike(0)
}, {min: 1});

However this generates the following pact file:

"body": [
          {
            "name": "Test",
            "documentId": 0,
            "documentCategoryId": 5,
            "contentLength": 0
          }
        ],
        "matchingRules": {
          "$.body": {
            "min": 1
          },
          "$.body[*].*": {
            "match": "type"
          },
          "$.body[*].name": {
            "match": "type"
          },
          "$.body[*].documentId": {
            "match": "type"
          },
          "$.body[*].documentCategoryId": {
            "match": "type"
          },
          "$.body[*].contentLength": {
            "match": "type"
          }
        }

This file is fine except for the expression: $.body.[]. which tries to match all fields, including the nullable ones which I did not write matchers for. This is causing the pact test to fail on the provider side as I am getting the error:
$.body.0.documentCategoryCode -> Expected null to be the same type as 0.

Why is the wild card matcher being added? Is this because eachLike expects an object as the root? Is there any know work arounds for this?

Any help would be much appreciated. Many thanks!

uglyog commented 6 years ago

The wildcard matcher is being added by the eachLike function in Pact-JS. But that does not explain why it is trying to compare the null value to a zero.

uglyog commented 6 years ago

Can you confirm the version of pact-jvm you are using, as I am unable to replicate this with the latest on master.

benilovp commented 6 years ago

JVM version: pact-jvm-provider-junit_2.11 version: '3.4.0 I assumed that 0 was a default matcher if the field in question is not mentioned in the JSON above the matching rules..

uglyog commented 6 years ago

No, the previous versions used to try do a type match against the first field in the map, which is silly because a map does not necessarily have key ordering. Versions 3.5.x+ changed to ignore those fields.

benilovp commented 6 years ago

I've tested this with pact jvm 3.5.8 and the matcher is no longer trying to match null fields. However now the wild card is causing my provider test to return successfully, even when a field is missing. For example:

"response": {
        "status": 200,
        "headers": {
          "content-type": "application/json"
        },
        "body": [
          {
            "doesNotExist": "Test",
            "documentId": 0,
          }
        ],
        "matchingRules": {
          "$.body": {
            "min": 1
          },
          "$.body[*].*": {
            "match": "type"
          },
          "$.body[*].doesNotExist": {
            "match": "type"
          },
          "$.body[*].documentId": {
            "match": "type"
          }
        }

The above test passes, even though my endpoint does not return a field called 'doesNotExist'. If I remove the wild card ("$.body[].") then the test correctly fails and complains that 'doesNotExist' is missing.

uglyog commented 6 years ago

You are being affected by #313, which is triggered by the $.body[*].* definition. There is a proposed update to the pact specification to resolve this: https://github.com/pact-foundation/pact-specification/issues/47

gvdp commented 6 years ago

Hi @uglyog I´m having the same (or at least a very similar) issue, but I don't think it's related to those 2 issues you reference.

If I take the already stated pact as an example, my test passes no matter what my provider returns in the array. For example both of these responses would pass the test:

[{
 "doesNotExist": "Test"
}]
[ {
  "doesNotExist": "Test",
  "documentId": "a string" 
}]

whereas I would expect both of these to fail because for the first it doesn't contain documentId and in the second example documentId is a string and not an integer.

If I manually remove the

"$.body[*].*": {
  "match": "type"
}

line from the contract the test does fail as I would expect it to.

It seems that the verifier only checks that the array contains something of type Object and is not verifying the specific properties of that object. Am I seeing this correct?