Message Pact verification results are incorrectly reported

[mefellows]

Given a message pact verification, where there are some successful and some unsuccessful verifications, they are incorrectly reported in Pactflow.

For example, given the following verification output:

Verifying a pact between MyJSMessageConsumer and MyJSMessageProvider

  a request for a dog
     Given a dog named drover
    generates a message which
      includes metadata
        "contentType" with value "application/json" (OK)
        "queue" with value "animals" (OK)
      has a matching body (OK)

  a request for a cat
     Given a cat named rover
    generates a message which
      includes metadata
        "contentType" with value "application/json" (OK)
        "queue" with value "animals" (FAILED)
      has a matching body (FAILED)

Failures:

1) Verifying a pact between MyJSMessageConsumer and MyJSMessageProvider Given a cat named rover - a request for a cat
    1.1) has a matching body
           $ -> Actual map is missing the following keys: id, name, type
    -{
  "id": 1,
  "name": "rover",
  "type": "bulldog"
}
    +{}

    1.2) has matching metadata
           Expected message metadata 'queue' to have value 'animals' but was ''

There were 1 pact failures

All interactions in the pact will be marked as failed in Pact Broker / Pactflow.

Software versions

• OS: Mac OSX 12.2.1
• Consumer Pact library: 10.4.0
• Provider Pact library: 10.4.0
• Node Version: 18.2.0

Issue Checklist

Please confirm the following:

• [x] I have upgraded to the latest
• [x] I have the read the FAQs in the Readme
• [x] I have triple checked, that there are no unhandled promises in my code and have read the section on intermittent test failures
• [x] I have set my log level to debug and attached a log file showing the complete request/response cycle
• [x] For bonus points and virtual high fives, I have created a reproduceable git repository (see below) to illustrate the problem

Expected behaviour

When there is a failure in a message pact, the failures are correctly surfaced to the Pact Broker / Pactflow.

Actual behaviour

All interactions are marked as failed, despite only some of them failing. Additionally, the failure reasons are not shown:

Steps to reproduce

Run the message pact example here: https://github.com/pact-foundation/pact-js/tree/repro/message-verification-results

# git clone ...
cd examples/messages
npm t

Relevant log files

iTerm2 Session 13 Dec 2022 at 1:46:34 pm.txt

It looks like the interactionId of the failure is missing, which could be why Pactflow isn't able to give information on which interaction is failing and why:

2022-12-13T02:46:12.491385Z DEBUG ThreadId(02) pact_verifier::pact_broker: Sending JSON to https://matt.pactflow.io/pacts/provider/MyJSMessageProvider/consumer/MyJSMessageConsumer/pact-version/427ca571c705443c2fd4488cd05a6db1c3b365de/metadata/c1tdW2JdPW1hc3RlciZzW11bbF09dHJ1ZSZzW11bY3ZdPTQ5/verification-results using POST: {"providerApplicationVersion":"10.3.1-repromessage-verification-results+50.3062ab6.SNAPSHOT.SB-AS-G7GM9F7","success":false,"testResults":[{"interactionId":"","mismatches":[{"attribute":"body","description":"Actual map is missing the following keys: id, name, type","identifier":"$"},{"attribute":"metadata","description":"Expected message metadata 'queue' but was missing","identifier":"queue"}],"success":false}],"verifiedBy":{"implementation":"Pact-Rust","version":"1.0.1"}}

[rholshausen]

Pact FFI Library 0.3.19 released to fix this

[mefellows]

Awesome, thanks. Will push out a change shortly!

[mefellows]

pact-core 13.3.2 has just gone out with a fix for this, which should come in transitively with an update to pact js. Could you please re-try and let us know how you go?

[mefellows]

Re testing it with the latest Pact JS seems to resolve it. Closing for now