Cannot verify message provider pact with v3 matchers

skaggmannen commented 4 years ago

Software versions

OS: Pop!_OS 19.10
Consumer Pact library: pact-jvm-consumer-maven-junit v3.5.3
Provider Pact library: pact-go v1.0.4
Golang Version: go version go1.13.5 linux/amd64

Golang environment:

GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/freall/.cache/go-build"
GOENV="/home/freall/.config/go/env"
GOEXE=""
GOFLAGS=" -mod="
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY="git.aah.assaabloy.net"
GONOSUMDB="git.aah.assaabloy.net"
GOOS="linux"
GOPATH="/home/freall/go"
GOPRIVATE="git.aah.assaabloy.net"
GOPROXY="direct"
GOROOT="/snap/go/current"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/snap/go/current/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/freall/Projects/vostio/km-key/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build208436467=/tmp/go-build -gno-record-gcc-switches"

Expected behaviour

It should be possible to verify a message pact containing v3 matching rules.

Actual behaviour

The matching rules are ignored and only exact matches are accepted.

Steps to reproduce

I wrote a message pact provider test using the example as baseline, but it seems like the framework is ignoring the matching rules in the pact file for some reason.

I tried verifying the following pact file:

{
    "consumer": {
        "name": "card-key-service"
    },
    "provider": {
        "name": "key-service"
    },
    "messages": [
        {
            "description": "key created event message",
            "metaData": {
                "Content-Type": "application/json; charset=UTF-8"
            },
            "contents": {
                "systemId": "dummy-system-id",
                "keyData": "dummy-key-data",
                "keyId": "6f802149-f7f7-4c85-8d72-0a3f8f9ea3e6",
                "type": "KeyCreated"
            },
            "providerStates": [
                {
                    "name": "key service is ready"
                }
            ],
            "matchingRules": {
                "body": {
                    "$.type": {
                        "matchers": [
                            {
                                "match": "regex",
                                "regex": "^KeyCreated*"
                            }
                        ],
                        "combine": "AND"
                    },
                    "$.systemId": {
                        "matchers": [
                            {
                                "match": "type"
                            }
                        ],
                        "combine": "AND"
                    },
                    "$.keyId": {
                        "matchers": [
                            {
                                "match": "type"
                            }
                        ],
                        "combine": "AND"
                    },
                    "$.keyData": {
                        "matchers": [
                            {
                                "match": "type"
                            }
                        ],
                        "combine": "AND"
                    }
                }
            }
        }
    ],
    "metadata": {
        "pact-specification": {
            "version": "3.0.0"
        },
        "pact-jvm": {
            "version": "3.5.3"
        }
    }
}

But I get the following errors:

2020/01/14 11:45:34 [INFO] checking pact-mock-service within range >= 3.1.0, < 4.0.0
2020/01/14 11:45:35 [INFO] checking pact-provider-verifier within range >= 1.23.0, < 3.0.0
2020/01/14 11:45:36 [INFO] checking pact-broker within range >= 1.18.0
--- FAIL: TestKeyEventProviderPact (3.58s)
=== RUN   TestKeyEventProviderPact/has_matching_content
    --- FAIL: TestKeyEventProviderPact/has_matching_content (0.00s)
        pact.go:601: Verifying a pact between card-key-service and key-service Given key service is ready key created event message has matching content
        pact.go:603: Actual: {"created":"2020-01-14T11:45:38+01:00","eventId":"677c4b05-d1ad-4386-9452-ad3a39083d2e","key":{"accessPolicies":[{"accessPointIds":[101,102],"accessPoints":"SPECIFIC","cancelKeys":["1c4d0d93-81d2-429f-b6c5-5c5e4055f91a","531c472b-c160-48b8-a765-51001d26ca92"],"deadboltPolicy":"DENY","effect":"GRANT","outputPolicy":"ALL","outputs":null}],"dataBlocks":["ZHVtbXktZGF0YQ=="],"keyId":"3b964e61-9709-45d2-8b81-9596acef5bf2","registrationNumber":123,"systemId":"79e899bb-2154-4e29-b7f5-b665ef3e6fd7","validFrom":"","validUntil":"2020-01-14T08:25:00Z"},"keyData":"eyJrZXlJZCI6IjNiOTY0ZTYxLTk3MDktNDVkMi04YjgxLTk1OTZhY2VmNWJmMiIsImtleURhdGFCbG9ja3MiOlsiNjQ3NTZkNmQ3OTJkNjQ2MTc0NjEiXSwidHlwZSI6IkxlZ2FjeUtleSJ9","keyId":"3b964e61-9709-45d2-8b81-9596acef5bf2","systemId":"79e899bb-2154-4e29-b7f5-b665ef3e6fd7","type":"KeyCreated"}

            Diff
            --------------------------------------
            Key: - is expected 
                 + is actual 
            Matching keys and values are not shown

             {
            -  "systemId": "dummy-system-id",
            -  "keyData": "dummy-key-data",
            -  "keyId": "6f802149-f7f7-4c85-8d72-0a3f8f9ea3e6"
            +  "systemId": "79e899bb-2154-4e29-b7f5-b665ef3e6fd7",
            +  "keyData": "eyJrZXlJZCI6IjNiOTY0ZTYxLTk3MDktNDVkMi04YjgxLTk1OTZhY2VmNWJmMiIsImtleURhdGFCbG9ja3MiOlsiNjQ3NTZkNmQ3OTJkNjQ2MTc0NjEiXSwidHlwZSI6IkxlZ2FjeUtleSJ9",
            +  "keyId": "3b964e61-9709-45d2-8b81-9596acef5bf2"
             }

            Description of differences
            --------------------------------------
            * Expected "dummy-system-id" but got "79e899bb-2154-4e29-b7f5-b665ef3e6fd7" at $.systemId
            * Expected "dummy-key-data" but got "eyJrZXlJZCI6IjNiOTY0ZTYxLTk3MDktNDVkMi04YjgxLTk1OTZhY2VmNWJmMiIsImtleURhdGFCbG9ja3MiOlsiNjQ3NTZkNmQ3OTJkNjQ2MTc0NjEiXSwidHlwZSI6IkxlZ2FjeUtleSJ9" at $.keyData
            * Expected "6f802149-f7f7-4c85-8d72-0a3f8f9ea3e6" but got "3b964e61-9709-45d2-8b81-9596acef5bf2" at $.keyId

        pact.go:604: Check to ensure that all message expectations have corresponding message handlers
    key_event_provider_pact_test.go:109: 
            Error Trace:    key_event_provider_pact_test.go:109
            Error:          Received unexpected error:
                            error verifying provider: exit status 1

                            STDERR:
                            INFO: Reading pact at ./provider-pacts/card-key-service-key-service.json
                            WARN: Ignoring unsupported matching rules {"matchers"=>[{"match"=>"regex", "regex"=>"^KeyCreated*"}], "combine"=>"AND"} for path $['type']
                            WARN: Ignoring unsupported matching rules {"matchers"=>[{"match"=>"type"}], "combine"=>"AND"} for path $['systemId']
                            WARN: Ignoring unsupported matching rules {"matchers"=>[{"match"=>"type"}], "combine"=>"AND"} for path $['keyId']
                            WARN: Ignoring unsupported matching rules {"matchers"=>[{"match"=>"type"}], "combine"=>"AND"} for path $['keyData']
                            WARN: Skipping set up for provider state 'key service is ready' for consumer 'card-key-service' as there is no --provider-states-setup-url specified.

                            STDOUT:
                            {"version":"3.9.1","examples":[{"id":"/home/freall/pact/lib/vendor/ruby/2.2.0/gems/pact-1.43.0/lib/pact/provider/rspec.rb[1:1:1:1]","description":"has matching content","full_description":"Verifying a pact between card-key-service and key-service Given key service is ready key created event message has matching content","status":"failed","file_path":"/home/freall/pact/lib/vendor/ruby/2.2.0/gems/pact-1.43.0/lib/pact/provider/rspec.rb","line_number":124,"run_time":0.033950855,"pending_message":null,"exception":{"class":"RSpec::Expectations::ExpectationNotMetError","message":"Actual: {\"created\":\"2020-01-14T11:45:38+01:00\",\"eventId\":\"677c4b05-d1ad-4386-9452-ad3a39083d2e\",\"key\":{\"accessPolicies\":[{\"accessPointIds\":[101,102],\"accessPoints\":\"SPECIFIC\",\"cancelKeys\":[\"1c4d0d93-81d2-429f-b6c5-5c5e4055f91a\",\"531c472b-c160-48b8-a765-51001d26ca92\"],\"deadboltPolicy\":\"DENY\",\"effect\":\"GRANT\",\"outputPolicy\":\"ALL\",\"outputs\":null}],\"dataBlocks\":[\"ZHVtbXktZGF0YQ==\"],\"keyId\":\"3b964e61-9709-45d2-8b81-9596acef5bf2\",\"registrationNumber\":123,\"systemId\":\"79e899bb-2154-4e29-b7f5-b665ef3e6fd7\",\"validFrom\":\"\",\"validUntil\":\"2020-01-14T08:25:00Z\"},\"keyData\":\"eyJrZXlJZCI6IjNiOTY0ZTYxLTk3MDktNDVkMi04YjgxLTk1OTZhY2VmNWJmMiIsImtleURhdGFCbG9ja3MiOlsiNjQ3NTZkNmQ3OTJkNjQ2MTc0NjEiXSwidHlwZSI6IkxlZ2FjeUtleSJ9\",\"keyId\":\"3b964e61-9709-45d2-8b81-9596acef5bf2\",\"systemId\":\"79e899bb-2154-4e29-b7f5-b665ef3e6fd7\",\"type\":\"KeyCreated\"}\n\nDiff\n--------------------------------------\nKey: \u001b[31m-\u001b[0m\u001b[31m is expected \n\u001b[0m\u001b[32m     +\u001b[0m\u001b[32m is actual \n\u001b[0mMatching keys and values are not shown\n\n\u001b[0m {\n\u001b[31m-  \"systemId\": \"dummy-system-id\",\n\u001b[0m\u001b[31m-  \"keyData\": \"dummy-key-data\",\n\u001b[0m\u001b[31m-  \"keyId\": \"6f802149-f7f7-4c85-8d72-0a3f8f9ea3e6\"\n\u001b[0m\u001b[32m+  \"systemId\": \"79e899bb-2154-4e29-b7f5-b665ef3e6fd7\",\n\u001b[0m\u001b[32m+  \"keyData\": \"eyJrZXlJZCI6IjNiOTY0ZTYxLTk3MDktNDVkMi04YjgxLTk1OTZhY2VmNWJmMiIsImtleURhdGFCbG9ja3MiOlsiNjQ3NTZkNmQ3OTJkNjQ2MTc0NjEiXSwidHlwZSI6IkxlZ2FjeUtleSJ9\",\n\u001b[0m\u001b[32m+  \"keyId\": \"3b964e61-9709-45d2-8b81-9596acef5bf2\"\n\u001b[0m }\n\nDescription of differences\n--------------------------------------\n* Expected \"dummy-system-id\" but got \"79e899bb-2154-4e29-b7f5-b665ef3e6fd7\" at $.systemId\n* Expected \"dummy-key-data\" but got \"eyJrZXlJZCI6IjNiOTY0ZTYxLTk3MDktNDVkMi04YjgxLTk1OTZhY2VmNWJmMiIsImtleURhdGFCbG9ja3MiOlsiNjQ3NTZkNmQ3OTJkNjQ2MTc0NjEiXSwidHlwZSI6IkxlZ2FjeUtleSJ9\" at $.keyData\n* Expected \"6f802149-f7f7-4c85-8d72-0a3f8f9ea3e6\" but got \"3b964e61-9709-45d2-8b81-9596acef5bf2\" at $.keyId\n","backtrace":["/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-support-3.9.2/lib/rspec/support.rb:97:in `block in <module:Support>'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-support-3.9.2/lib/rspec/support.rb:106:in `call'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-support-3.9.2/lib/rspec/support.rb:106:in `notify_failure'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-expectations-3.9.0/lib/rspec/expectations/fail_with.rb:35:in `fail_with'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-expectations-3.9.0/lib/rspec/expectations/handler.rb:40:in `handle_failure'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-expectations-3.9.0/lib/rspec/expectations/handler.rb:50:in `block in handle_matcher'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-expectations-3.9.0/lib/rspec/expectations/handler.rb:27:in `with_matcher'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-expectations-3.9.0/lib/rspec/expectations/handler.rb:48:in `handle_matcher'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-expectations-3.9.0/lib/rspec/expectations/expectation_target.rb:65:in `to'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/pact-1.43.0/lib/pact/provider/rspec.rb:129:in `block in describe_message'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:257:in `instance_exec'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:257:in `block in run'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:503:in `block in with_around_and_singleton_context_hooks'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:460:in `block in with_around_example_hooks'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/hooks.rb:472:in `block in run'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/hooks.rb:610:in `run_around_example_hooks_for'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/hooks.rb:472:in `run'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:460:in `with_around_example_hooks'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:503:in `with_around_and_singleton_context_hooks'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:254:in `run'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:633:in `block in run_examples'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:629:in `map'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:629:in `run_examples'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:595:in `run'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:596:in `block in run'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:596:in `map'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:596:in `run'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:596:in `block in run'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:596:in `map'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:596:in `run'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/runner.rb:121:in `block (3 levels) in run_specs'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/runner.rb:121:in `map'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/runner.rb:121:in `block (2 levels) in run_specs'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/configuration.rb:2031:in `with_suite_hooks'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/runner.rb:116:in `block in run_specs'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/reporter.rb:74:in `report'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/runner.rb:115:in `run_specs'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/runner.rb:89:in `run'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/rspec-core-3.9.1/lib/rspec/core/runner.rb:71:in `run'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/pact-1.43.0/lib/pact/provider/pact_spec_runner.rb:89:in `run_specs'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/pact-1.43.0/lib/pact/provider/pact_spec_runner.rb:35:in `run'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/pact-1.43.0/lib/pact/cli/run_pact_verification.rb:67:in `run_with_pact_uri_object'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/pact-1.43.0/lib/pact/cli/run_pact_verification.rb:50:in `run_specs'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/pact-1.43.0/lib/pact/cli/run_pact_verification.rb:21:in `call'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/pact-1.43.0/lib/pact/cli/run_pact_verification.rb:14:in `call'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/pact-provider-verifier-1.26.0/lib/pact/provider_verifier/app.rb:161:in `verify_pact'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/pact-provider-verifier-1.26.0/lib/pact/provider_verifier/app.rb:41:in `block in call'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/pact-provider-verifier-1.26.0/lib/pact/provider_verifier/app.rb:40:in `collect'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/pact-provider-verifier-1.26.0/lib/pact/provider_verifier/app.rb:40:in `call'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/pact-provider-verifier-1.26.0/lib/pact/provider_verifier/app.rb:32:in `call'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/pact-provider-verifier-1.26.0/lib/pact/provider_verifier/cli/verify.rb:53:in `verify'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/thor-0.20.3/lib/thor/base.rb:466:in `start'","/home/freall/pact/lib/vendor/ruby/2.2.0/gems/pact-provider-verifier-1.26.0/lib/pact/provider_verifier/cli/custom_thor.rb:17:in `start'","/home/freall/pact/lib/app/pact-provider-verifier.rb:33:in `<main>'"]}}],"summary":{"duration":0.036610422,"example_count":1,"failure_count":1,"pending_count":0,"errors_outside_of_examples_count":0},"summary_line":"1 example, 1 failure"}
            Test:           TestKeyEventProviderPact
FAIL

Process finished with exit code 1

Relevent log files

Please ensure you set logging to DEBUG and attach any relevant log files here (or link from a gist).

mefellows commented 4 years ago

Thanks for the detailed report @skaggmannen. We currently only support v2 matchers. We are aiming for v3 support in q2 this year, so hopefully that gives you an indication of time-frames.

skaggmannen commented 4 years ago

Does this mean that you do not actually support message pacts since they use v3 of the pact dsl? How about the examples for message pacts included in the project?

14 jan. 2020 kl. 12:46 skrev Matt Fellows notifications@github.com:

Thanks for the detailed report @skaggmannen. We currently only support v2 matchers. We are aiming for v3 support in q2 this year, so hopefully that gives you an indication of time-frames.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

mefellows commented 4 years ago

It's a bit confusing. We support just parsing the v3 spec file and the message functionality, but not the v3 matchers.

See https://docs.pact.io/feature_support

On Wed, Jan 15, 2020, 4:03 AM Fredrik Allansson notifications@github.com wrote:

Does this mean that you do not actually support message pacts since they use v3 of the pact dsl? How about the examples for message pacts included in the project?

14 jan. 2020 kl. 12:46 skrev Matt Fellows notifications@github.com:

Thanks for the detailed report @skaggmannen. We currently only support v2 matchers. We are aiming for v3 support in q2 this year, so hopefully that gives you an indication of time-frames.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pact-foundation/pact-go/issues/123?email_source=notifications&email_token=AAANFDASYX7URNS4WH3CSH3Q5XV7JA5CNFSM4KGRAATKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEI5L3PY#issuecomment-574275007, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAANFDFC36TRD5WADWKZFDDQ5XV7JANCNFSM4KGRAATA .

mefellows commented 1 year ago

V3 is now supported in 2.x.x, closing.

pact-foundation / pact-go