search

pact-foundation / pact-js

JS version of Pact. Pact is a contract testing framework for HTTP APIs and non-HTTP asynchronous messaging systems.
https://pact.io
Other
1.59k stars 343 forks source link

Error in native callback #1090

Closed druoid closed 1 year ago

druoid commented 1 year ago

Software versions

Issue Checklist

Please confirm the following:

Expected behaviour

Test should pass when publishing pacts to broker (which it does do but test fails)

Actual behaviour

Test fails even though pacts are published to broker

Steps to reproduce

Clone repo https://github.com/druoid/pact-js-poc Navigate to examples > v3 > typescript folder npm run test

Relevant log files

`andrewchambers@Andrews-MacBook-Pro typescript % npm run test

typescript-pact-v3@1.0.0 test nyc --check-coverage --reporter=html --reporter=text-summary mocha test/*.spec.ts

The Users API (consumer) get /users/:id 2023-05-10T00:20:32.811713Z DEBUG ThreadId(01) pact_plugin_driver::catalogue_manager: Updated catalogue entries: core/transport/http core/transport/https 2023-05-10T00:20:32.811765Z DEBUG ThreadId(01) pact_plugin_driver::catalogue_manager: Updated catalogue entries: core/content-generator/binary core/content-generator/json core/content-matcher/json core/content-matcher/multipart-form-data core/content-matcher/text core/content-matcher/xml 2023-05-10T00:20:32.811791Z DEBUG ThreadId(01) pact_plugin_driver::catalogue_manager: Updated catalogue entries: core/matcher/v1-equality core/matcher/v2-max-type core/matcher/v2-min-type core/matcher/v2-minmax-type core/matcher/v2-regex core/matcher/v2-type core/matcher/v3-content-type core/matcher/v3-date core/matcher/v3-datetime core/matcher/v3-decimal-type core/matcher/v3-includes core/matcher/v3-integer-type core/matcher/v3-null core/matcher/v3-number-type core/matcher/v3-time core/matcher/v4-array-contains core/matcher/v4-equals-ignore-order core/matcher/v4-max-equals-ignore-order core/matcher/v4-min-equals-ignore-order core/matcher/v4-minmax-equals-ignore-order core/matcher/v4-not-empty core/matcher/v4-semver 2023-05-10T00:20:32.812059Z DEBUG ThreadId(01) pact_mock_server::mock_server: Started mock server on 127.0.0.1:56365 2023-05-10T00:20:32.816098Z DEBUG tokio-runtime-worker hyper::proto::h1::io: parsed 4 headers 2023-05-10T00:20:32.816114Z DEBUG tokio-runtime-worker hyper::proto::h1::conn: incoming body is empty 2023-05-10T00:20:32.816121Z DEBUG tokio-runtime-worker pact_mock_server::hyper_server: Creating pact request from hyper request 2023-05-10T00:20:32.816124Z DEBUG tokio-runtime-worker pact_mock_server::hyper_server: Extracting query from uri /users/1 2023-05-10T00:20:32.816146Z INFO tokio-runtime-worker pact_mock_server::hyper_server: Received request HTTP Request ( method: GET, path: /users/1, query: None, headers: Some({"accept": ["application/json"], "connection": ["close"], "host": ["127.0.0.1:56365"], "user-agent": ["axios/0.27.2"]}), body: Empty ) 2023-05-10T00:20:32.816183Z INFO tokio-runtime-worker pact_matching: comparing to expected HTTP Request ( method: GET, path: /users/1, query: None, headers: None, body: Missing ) 2023-05-10T00:20:32.816187Z DEBUG tokio-runtime-worker pact_matching: body: '' 2023-05-10T00:20:32.816189Z DEBUG tokio-runtime-worker pact_matching: matching_rules: MatchingRules { rules: {PATH: MatchingRuleCategory { name: PATH, rules: {} }} } 2023-05-10T00:20:32.816192Z DEBUG tokio-runtime-worker pact_matching: generators: Generators { categories: {} } 2023-05-10T00:20:32.816205Z DEBUG tokio-runtime-worker pact_matching::matchers: String -> String: comparing '/users/1' to '/users/1' ==> true cascaded=false matcher=Equality 2023-05-10T00:20:32.816275Z DEBUG tokio-runtime-worker pact_matching: expected content type = '/', actual content type = '/' 2023-05-10T00:20:32.816296Z DEBUG tokio-runtime-worker pact_matching: content type header matcher = 'RuleList { rules: [], rule_logic: And, cascaded: false }' 2023-05-10T00:20:32.816305Z DEBUG tokio-runtime-worker pact_matching: --> Mismatches: [] 2023-05-10T00:20:32.816332Z DEBUG tokio-runtime-worker pact_mock_server::hyper_server: Test context = {"mockServer": Object {"href": String("http://127.0.0.1:56365"), "port": Number(56365)}} 2023-05-10T00:20:32.816343Z INFO tokio-runtime-worker pact_mock_server::hyper_server: Request matched, sending response HTTP Response ( status: 200, headers: Some({"Content-Type": ["application/json"]}), body: Present(40 bytes, application/json) ) 2023-05-10T00:20:32.816347Z DEBUG tokio-runtime-worker pact_mock_server::hyper_server: body: '{"age":39,"id":1,"name":"Homer Simpson"}' 2023-05-10T00:20:32.816374Z DEBUG tokio-runtime-worker hyper::proto::h1::io: flushed 392 bytes ✔ returns the requested user (consumer) Pact Verification [2023-05-10 00:20:32.820 +0000] INFO (69083 on Andrews-MacBook-Pro.local): pact-node@10.17.7: Verifying Pact Files [12:20:32.749] DEBUG (69083): pact-core@13.13.5: Initalising native core at log level 'debug' [12:20:32.809] DEBUG (69083): pact@10.1.2: free port discovered: 56365 [12:20:32.811] DEBUG (69083): pact@10.1.2: Setting up Pact mock server with Consumer "User Web" and Provider "User API" using mock service on Port: "56365" [12:20:32.812] DEBUG (69083): pact@10.1.2: mock service started on port: 56365 2023-05-10T00:20:33.909547Z DEBUG tokio-runtime-worker hyper::proto::h1::io: parsed 4 headers 2023-05-10T00:20:33.909564Z DEBUG tokio-runtime-worker hyper::proto::h1::conn: incoming body is empty 2023-05-10T00:20:33.909570Z DEBUG tokio-runtime-worker pact_mock_server::hyper_server: Creating pact request from hyper request 2023-05-10T00:20:33.909573Z DEBUG tokio-runtime-worker pact_mock_server::hyper_server: Extracting query from uri /users/1 2023-05-10T00:20:33.909590Z INFO tokio-runtime-worker pact_mock_server::hyper_server: Received request HTTP Request ( method: GET, path: /users/1, query: None, headers: Some({"host": ["127.0.0.1:56365"], "cookie": [""], "x-forwarded-for": ["127.0.0.1"], "content-length": ["0"]}), body: Empty ) 2023-05-10T00:20:33.909617Z INFO tokio-runtime-worker pact_matching: comparing to expected HTTP Request ( method: GET, path: /users/1, query: None, headers: None, body: Missing ) 2023-05-10T00:20:33.909619Z DEBUG tokio-runtime-worker pact_matching: body: '' 2023-05-10T00:20:33.909621Z DEBUG tokio-runtime-worker pact_matching: matching_rules: MatchingRules { rules: {PATH: MatchingRuleCategory { name: PATH, rules: {} }} } 2023-05-10T00:20:33.909624Z DEBUG tokio-runtime-worker pact_matching: generators: Generators { categories: {} } 2023-05-10T00:20:33.909632Z DEBUG tokio-runtime-worker pact_matching::matchers: String -> String: comparing '/users/1' to '/users/1' ==> true cascaded=false matcher=Equality 2023-05-10T00:20:33.909708Z DEBUG tokio-runtime-worker pact_matching: expected content type = '/', actual content type = '/' 2023-05-10T00:20:33.909723Z DEBUG tokio-runtime-worker pact_matching: content type header matcher = 'RuleList { rules: [], rule_logic: And, cascaded: false }' 2023-05-10T00:20:33.909729Z DEBUG tokio-runtime-worker pact_matching: --> Mismatches: [] 2023-05-10T00:20:33.909752Z DEBUG tokio-runtime-worker pact_mock_server::hyper_server: Test context = {"mockServer": Object {"href": String("http://127.0.0.1:56365"), "port": Number(56365)}} 2023-05-10T00:20:33.909760Z INFO tokio-runtime-worker pact_mock_server::hyper_server: Request matched, sending response HTTP Response ( status: 200, headers: Some({"Content-Type": ["application/json"]}), body: Present(40 bytes, application/json) ) 2023-05-10T00:20:33.909765Z DEBUG tokio-runtime-worker pact_mock_server::hyper_server: body: '{"age":39,"id":1,"name":"Homer Simpson"}' 2023-05-10T00:20:33.909791Z DEBUG tokio-runtime-worker hyper::proto::h1::io: flushed 392 bytes 2023-05-10T00:20:33.910285Z DEBUG tokio-runtime-worker hyper::proto::h1::conn: read eof [2023-05-10 00:20:34.360 +0000] INFO (69083 on Andrews-MacBook-Pro.local): pact-node@10.17.7: Pact Verification succeeded. ✔ validates the expectations of UserService (1542ms) [2023-05-10 00:20:34.362 +0000] INFO (69083 on Andrews-MacBook-Pro.local): pact-node@10.17.7: Publishing pacts to broker at: https://fluxpoc.pactflow.io [2023-05-10 00:20:35.571 +0000] INFO (69083 on Andrews-MacBook-Pro.local): pact-node@10.17.7: Updated User Web version 1.0.1 Next steps: Configure the version branch to be the value of your repository branch. Pact successfully republished for User Web version 1.0.1 and provider User API with no content changes. View the published pact at https://fluxpoc.pactflow.io/pacts/provider/User%20API/consumer/User%20Web/version/1.0.1 Events detected: contract_published Next steps:

TimothyJones commented 1 year ago

You're calling the consumer lifecycle methods in a provider test. These:

  before(() => {
    return provider.setup();
  });

  afterEach(() => {
    return provider.verify();
  });

  after(() => {
    return provider.finalize();
  });

Are not valid for this test:

      return new Publisher(opts).publish();

Pact isn't detecting this misuse, and is passing the lifecycle calls on to the native core, which is failing because it isn't expecting those calls at that time. It's only a bug in the sense that it's crashing instead of failing with a "you can't call this method at this time".

Those lifecycle methods are valid for provider.addInteraction({...}), but that's part of a consumer test, not a provider test. You should keep those separate.

While we're here: your consumer spec file is also calling the lifecycle methods out of order, but in a different way.

You can't call provider.finalize(); without calling provider.verify() first (it should be called after the request has been sent).

You might find the provider.executeTest DSL more convenient, as it does a lot of this for you. Have a look in the documentation for examples.

await new Verifier(opts).verify();

^ Also, this doesn't make sense in the consumer test, this should be in the provider test.

If this answer doesn't make sense, have a read of this document that explains the difference between consumer and provider tests: https://docs.pact.io/getting_started/how_pact_works

druoid commented 1 year ago

Thanks @TimothyJones note to self don't try use chatgpt to generate tests.

TimothyJones commented 1 year ago

....you asked chat gpt for test code, and when it didn't work, you thought it was a bug in the library?!

Chat GPT is very impressive, but it is not there yet.

mefellows commented 1 year ago

Yep! Ask ChatGPT to divide 2134102837123987.98213 / 123123908712389712.123123 (I just made these numbers up). You'll see that has absolutely no idea how to answer it. It doesn't actually understand the concept of division.

mefellows commented 1 year ago

...but yeah, it's still pretty bloody amazing!

mefellows commented 1 year ago

Closing as wontfix. The v4 interface prevents lifecycle methods from being called out of order, I'd accept a PR that adds the appropriate guards to V3, but I think it would be messy to implement.