pact-foundation / pact-jvm

JVM version of Pact. Enables consumer driven contract testing, providing a mock service and DSL for the consumer project, and interaction playback and verification for the service provider project.
https://docs.pact.io
Apache License 2.0
1.08k stars 479 forks source link

The provider verification doesn't fail when the value for a generated body attribute wasn't set #1059

Open sdobrovolschi opened 4 years ago

sdobrovolschi commented 4 years ago

When a generated body attribute is used in the definition of an interaction, the TypeMatcher successfully matches the response body even if the expected value was not provided in the state callback.

Dsl interaction:

.body(new PactDslJsonBody()
     .valueFromProviderState("accountId", "${accountId}", "4beb44f1-53f7-4281-a78b-12c06d682067"))

Pact fragment:

"response": {
        "status": 200,
        "body": {
          "accountId": "4beb44f1-53f7-4281-a78b-12c06d682067"
        },
        "matchingRules": {
          "body": {
            "$.accountId": {
              "matchers": [
                {
                  "match": "type"
                }
              ],
              "combine": "AND"
            }
          }
        },
        "generators": {
          "body": {
            "$.accountId": {
              "type": "ProviderState",
              "expression": "${accountId}"
            }
          }
        }
      }

The expected failure should be:

BodyComparisonResult(mismatches={$.accountId=[BodyMismatch(expected="null", actual="af45e0ca-22dd-41fb-8d20-d4494df34e13", mismatch=Expected "null" but received "af45e0ca-22dd-41fb-8d20-d4494df34e13", path=$.accountId, diff=null)]}, diff=[{, -  "accountId": "null", +  "accountId": "af45e0ca-22dd-41fb-8d20-d4494df34e13", }])

The TypeMatcher has been added in the following patch: https://github.com/DiUS/pact-jvm/commit/80aa4db43775a3524274eba11d8eb19ce01deb71

Please note that for a generated response header attribute, no matchers are added and provider verification works as expected.

uglyog commented 4 years ago

The issue here is that the expression "${accountId}" becomes the string "null", which has the correct type.

It would be better to use a matcher that checks for a UUID, instead of any string.

The other thing to do is not use an expression, just the key. That will get the desired result.

I.e,

.body(new PactDslJsonBody()
     .valueFromProviderState("accountId", "accountId", "4beb44f1-53f7-4281-a78b-12c06d682067"))

results in

BodyComparisonResult(mismatches={$.accountId=[BodyMismatch(expected=null, actual="4beb44f1-53f7-4281-abcd-12c06d682067", mismatch=Expected "4beb44f1-53f7-4281-abcd-12c06d682067" to be null, path=$.accountId, diff=null)]}, diff=[{, -  "accountId": null, +  "accountId": "4beb44f1-53f7-4281-abcd-12c06d682067", }])
sdobrovolschi commented 4 years ago

Thank you @uglyog for your help.

I'm not sure the UUID matcher is an option because I'd like to assert the equality between the expected and the actual field.

The suggested option with the key addresses the issue, I mean when no value is provided then verification fails, but when a value is provided it's not injected and verification still fails when it should pass.

sdobrovolschi commented 4 years ago

I figured out why it wasn't injected - the interaction was expecting a String but the value from the provider sate was a UUID.

The interesting thing is if I change the interaction to expect a UUID e.g.

new PactDslJsonBody()
  .valueFromProviderState("accountId", "accountId", UUID.fromString("4beb44f1-53f7-4281-a78b-12c06d682067")))

and have an UUID value from the provider state then the value is not injected and verification fails, but I the value is a String then verification passes.

Also, it uses the TypeMatcher which is not what I'm expecting.

When I remove the

"matchingRules": {
          "body": {
            "$.accountId": {
              "matchers": [
                {
                  "match": "type"
                }
              ],
              "combine": "AND"
            }
          }
        },

then the equality is asserted.