FFI > Using `values` matcher for header: It shouldn't return mismatches about 2 identical strings are not equal

tienvx commented 8 months ago

I'm trying to match header to contain multiple values using ffi call but it doesn't work as I expected. Let focus on values matcher first:

FFI call:

pactffi_with_header_v2($interactionId, $ffi->InteractionPart_Response, 'X-IMPROVED', 0, '{"value":["like","regex","values","arrayContaining"],"pact:matcher:type":"values"}');

Trimmed pact file:

{
  "consumer": {
    "name": "matchersConsumer"
  },
  "interactions": [
    {
      "description": "A get request to /matchers",
      "request": {},
      "response": {
        "body": {},
        "headers": {
          "X-IMPROVED": [
            "like",
            "regex",
            "values",
            "arrayContaining"
          ]
        },
        "matchingRules": {
          "header": {
            "X-IMPROVED": {
              "combine": "AND",
              "matchers": [
                {
                  "match": "values"
                }
              ]
            }
          }
        },
        "status": 200
      },
      "transport": "http",
      "type": "Synchronous/HTTP"
    }
  ],
  "metadata": {
    "pactSpecification": {
      "version": "4.0"
    }
  },
  "provider": {
    "name": "matchersProvider"
  }
}

Provider response's header (same problem in 2 cases):

Single header multiple values

X-IMPROVED: regex, like, values, arrayContaining

Multi headers with same name, different values

X-IMPROVED: like
X-IMPROVED: regex
X-IMPROVED: values
X-IMPROVED: arrayContaining

Verifier logs (same log in 2 cases)

2024-01-19T15:16:50.632176Z DEBUG ThreadId(01) verify_interaction{interaction="A get request to /matchers"}: pact_matching::matchers: String -> String: comparing 'like' to 'like' ==> false cascaded=false matcher=Values
2024-01-19T15:16:50.632185Z DEBUG ThreadId(01) verify_interaction{interaction="A get request to /matchers"}: pact_matching::matchers: String -> String: comparing 'regex' to 'regex' ==> false cascaded=false matcher=Values
2024-01-19T15:16:50.632190Z DEBUG ThreadId(01) verify_interaction{interaction="A get request to /matchers"}: pact_matching::matchers: String -> String: comparing 'values' to 'values' ==> false cascaded=false matcher=Values
2024-01-19T15:16:50.632197Z DEBUG ThreadId(01) verify_interaction{interaction="A get request to /matchers"}: pact_matching::matchers: String -> String: comparing 'arrayContaining' to 'arrayContaining' ==> false cascaded=false matcher=Values

Expected: No mismatches for X-IMPROVED

Actual: Weird message from provider's verifier, saying that 2 identical strings are not equal (similar message in both cases):

Failures:

1) Verifying a pact between matchersConsumer and matchersProvider Given Get Matchers - A get request to /matchers
    1.1) includes header 'X-IMPROVED' with value 'like'
           Expected header 'X-IMPROVED' to have value 'like' but was 'like'
           Expected header 'X-IMPROVED' to have value 'regex' but was 'regex'
           Expected header 'X-IMPROVED' to have value 'values' but was 'values'
           Expected header 'X-IMPROVED' to have value 'arrayContaining' but was 'arrayContaining'

For more information:

Here is the code (incomplete, please ignore code that is not X-IMPROVED) https://github.com/tienvx/pact-php/commit/2993bcec74a21df847f483186fd0d1529a629908
See Slack's thread for more information https://pact-foundation.slack.com/archives/C02BXLDJ7JR/p1698024661692909

rholshausen commented 8 months ago

There are a few problems here:

The matching expression $['X-IMPROVED'] is not correct, it should be X-IMPROVED. This is probably an issue in the processing of the intermediate JSON format.
It's displaying the wrong error message. The actual message from the matching engine is Mismatch with header 'X-IMPROVED': Unable to match 'like' using Values for value at index 0. This is a bigger issue with the description generated by the verification process.

sveko-code commented 5 months ago

We're having the same problem, but with any matchers for headers.

Frameworks / Versions used: Consumer: Angular

@pact-foundation/pact: 12.1.2
jest-pact: 0.11.0

Consumer-test (rough excerpt):

...
await provider.addInteraction({
    uponReceiving: 'a request',
    withRequest: {
        method: 'GET',
        path: `/api/v1/data`,
        headers: {}
    },
    willRespondWith: {
        status: 200,
        headers: {
            'Content-Type': string('application/example')
        },
        body: {'data': 'someData'}
    }
} as InteractionObject);
...

generates (excerpt):

...
  "interactions": [
    {
      "description": "a request",
      "pending": false,
      "request": {
        "method": "GET",
        "path": "/api/v1/data"
      },
      "response": {
        "body": {
          "data": "someData"
        },
        "headers": {
          "Content-Type": [
            "application/example"
          ]
        },
        "matchingRules": {
          "header": {
            "$['Content-Type'][0]": {
              "combine": "AND",
              "matchers": [
                {
                  "match": "type"
                }
              ]
            }
          }
        },
        "status": 200
      },
      "type": "Synchronous/HTTP"
    }
  ]
  ...

We tried different matchers (string, regex, like, providerState), but all lead to the same problem.

After changing the key manually to Content-Type within the matchingRules, the provider-tests recognizes the matcher and passes.

Am i right that this is the same issue? Should i report this at another repo?

rholshausen commented 5 months ago

This is probably caused by #355

tienvx commented 5 months ago

I would like to test this again on the version 0.4.12, but this error preventing me:

Verifying a pact between matchersConsumer and matchersProvider
WARNING: Pact file has no interactions

This error doesn't happen on 0.4.11.

It look like that this error not related to this issue. If so I will create new issue for it.

UPDATED: this is the issue https://github.com/pact-foundation/pact-reference/issues/359

sveko-code commented 5 months ago

We're having the same problem, but with any matchers for headers.

Frameworks / Versions used: Consumer: Angular

@pact-foundation/pact: 12.1.2

jest-pact: 0.11.0

Consumer-test (rough excerpt):
...
await provider.addInteraction({
    uponReceiving: 'a request',
    withRequest: {
        method: 'GET',
        path: `/api/v1/data`,
        headers: {}
    },
    willRespondWith: {
        status: 200,
        headers: {
            'Content-Type': string('application/example')
        },
        body: {'data': 'someData'}
    }
} as InteractionObject);
...
generates (excerpt):
...
  "interactions": [
    {
      "description": "a request",
      "pending": false,
      "request": {
        "method": "GET",
        "path": "/api/v1/data"
      },
      "response": {
        "body": {
          "data": "someData"
        },
        "headers": {
          "Content-Type": [
            "application/example"
          ]
        },
        "matchingRules": {
          "header": {
            "$['Content-Type'][0]": {
              "combine": "AND",
              "matchers": [
                {
                  "match": "type"
                }
              ]
            }
          }
        },
        "status": 200
      },
      "type": "Synchronous/HTTP"
    }
  ]
  ...
We tried different matchers (string, regex, like, providerState), but all lead to the same problem.

After changing the key manually to Content-Type within the matchingRules, the provider-tests recognizes the matcher and passes.

Am i right that this is the same issue? Should i report this at another repo?

after updating pact-core my generated files are fine.

tienvx commented 5 months ago

This specific issue (using values matcher with header) still happen when I updated to 0.4.13

pact-foundation / pact-reference

FFI > Using `values` matcher for header: It shouldn't return mismatches about 2 identical strings are not equal #331