Update to ruby standalone 1.77.0

[bethesque]

Latest format (let me know if you need any changes):

{
  "version": "3.9.1",
  "examples": [
    {
      "id": "/redacted/.gem/ruby/2.4.5/gems/pact-1.46.1/lib/pact/provider/rspec.rb[1:1:1:1:1]",
      "description": "has status code 200",
      "full_description": "Verifying a pact between Foo and Bar [PENDING] A retrieve thing request with GET /thing returns a response which has status code 200",
      "status": "pending",
      "file_path": "/redacted/.gem/ruby/2.4.5/gems/pact-1.46.1/lib/pact/provider/rspec.rb",
      "line_number": 155,
      "run_time": 0.023331,
      "mismatches": []
    },
    {
      "id": "/redacted/.gem/ruby/2.4.5/gems/pact-1.46.1/lib/pact/provider/rspec.rb[1:1:1:1:3]",
      "description": "has a matching body",
      "full_description": "Verifying a pact between Foo and Bar [PENDING] A retrieve thing request with GET /thing returns a response which has a matching body",
      "status": "pending",
      "file_path": "/redacted/.gem/ruby/2.4.5/gems/pact-1.46.1/lib/pact/provider/rspec.rb",
      "line_number": 174,
      "run_time": 0.000922,
      "mismatches": [
        "Could not find key \"name\" (keys present are: a) at $"
      ]
    },
    {
      "id": "/redacted/.gem/ruby/2.4.5/gems/pact-1.46.1/lib/pact/provider/rspec.rb[1:1:1:1:2:1]",
      "description": "\"Content-Type\" which equals \"application/json\"",
      "full_description": "Verifying a pact between Foo and Bar [PENDING] A retrieve thing request with GET /thing returns a response which includes headers \"Content-Type\" which equals \"application/json\"",
      "status": "passed",
      "file_path": "/redacted/.gem/ruby/2.4.5/gems/pact-1.46.1/lib/pact/provider/rspec.rb",
      "line_number": 164,
      "run_time": 0.000137,
      "mismatches": []
    }
  ],
  "summary": {
    "duration": 0.025587,
    "example_count": 3,
    "failure_count": 0,
    "pending_count": 2,
    "errors_outside_of_examples_count": 0,
    "notices": [
      {
        "text": "This pact is being verified because it is the latest pact between Foo and Bar."
      },
      {
        "text": "This pact is in pending state because it has not yet been successfully verified by any versions of Bar with tag 'pdev' and 'foo'. If this verification fails, it will not cause the overall build to fail. Read more at https://pact.io/pending"
      }
    ]
  },
  "summary_line": "3 examples, 0 failures, 2 pending"
}

Note that the notices are objects, not strings, as in a previous example. I'll update the broker to put the URL of the pact in the notices.

[mefellows]

Awesome, thanks Beth!

[bethesque]

I'm wondering about including notices for success and failure scenarios, like this (this is the response from the broker, the JSON returned from the pact-verifier would already have the correct messages selected):

"notices": [
      {
        "timing": "pre_verification",
        "text": "This pact is being verified because it is the latest pact between Foo and Bar."
      },
      {
        "timing": "pre_verification",
        "text": "This pact is in pending state because it has not yet been successfully verified by any version of Bar with tag 'master'. If this verification fails, it will not cause the overall build to fail. Read more at https://pact.io/pending"
      },{
         "timing": "post_verification:failure",
         "text": "This pact is still in pending state for the tag 'master' as it has not yet been successfully verified by a version of Bar with the tag 'master'"
     },{
         "timing": "post_verification:success:publish_verification_results_disabled",
         "text": "This pact is still in pending state for the tag 'master' as the successful verification result has not yet been published"
     }, {
         "timing": "post_verification:success:post_verification_results_publication",
         "text": "This pact is no longer pending for the tag 'master' as the successful verification results have been published. If verification by a version tagged with 'master' fails in the future, it will cause the overall build to fail."
     }
    ]

Thoughts?

[mefellows]

Interesting. I mean, it certainly wouldn't hurt. How would you envisage us printing it in a test output?

[bethesque]

Before - pending

After - failed, still pending

After - succeeded, no longer pending

I think it would help people work out what was going on with the whole pending flow, which I am sure is going to confuse people.

[bethesque]

The JSON formatter would work out the right after verification notices to display, and put them into the JSON output as

"notices": [
      {
         "when": "before_verification",
         "text": "..."
     },{
         "when": "after_verification",
         "text": "..."
     }
    ]

and the go test harness could print them out at the appropriate stage.

[bethesque]

Actually, I should change "when" to something else given it's a key word.

[mefellows]

I think it would help people work out what was going on with the whole pending flow, which I am sure is going to confuse people.

Yep, I like this idea. I think I can make it work, but even if not, it might be helpful in other implementations.

Actually, I should change "when" to something else given it's a key word.

Perhaps "phase"?

If possible, I think it would be useful for each of the JSON objects in the JSONL response to have a top-level key for the source of the pact, the name of the consumer and also the tag used to fetch it. This way we can group and programmatically manipulate the output:

{
  "version": "3.9.1",
   "source": "https://pactbroker/.....",
   "tag": "branch:feat/something",
   "consumer": "consumer-name",
  "examples": [
    {
      "id": "/redacted/.gem/ruby/2.4.5/gems/pact-1.46.1/lib/pact/provider/rspec.rb[1:1:1:1:1]",
      "description": "has status code 200",
      "full_description": "Verifying a pact between Foo and Bar [PENDING] A retrieve thing request with GET /thing returns a response which has status code 200",
      "status": "pending",
      "file_path": "/redacted/.gem/ruby/2.4.5/gems/pact-1.46.1/lib/pact/provider/rspec.rb",
      "line_number": 155,
      "run_time": 0.023331,
      "mismatches": []
    },
    ...
  ],
  "summary": {
    "duration": 0.025587,
    "example_count": 3,
    "failure_count": 0,
    "pending_count": 2,
    "errors_outside_of_examples_count": 0,
    "notices": [
      {
        "text": "This pact is being verified because it is the latest pact between Foo and Bar."
      },
      {
        "text": "This pact is in pending state because it has not yet been successfully verified by any versions of Bar with tag 'pdev' and 'foo'. If this verification fails, it will not cause the overall build to fail. Read more at https://pact.io/pending"
      }
    ]
  },
  "summary_line": "3 examples, 0 failures, 2 pending"
}

[bethesque]

Easily done. The tag would need to be an array of tags, as it could be the latest of multiple tags - most commonly master and production.

[bethesque]

I can put the pending message in each interaction, but I think, as you mentioned before, that it would be a lot of noise as it's quite long, and printing the message once would be more helpful.

[mefellows]

Sounds ace (I thought tags might have been an array, this works just fine)

On Thu, Jan 23, 2020, 5:13 PM Beth Skurrie notifications@github.com wrote:

I can put the pending message in each interaction, but I think, as you mentioned before, that it would be a lot of noise as it's quite long, and printing the message once would be more helpful.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/pact-foundation/pact-go/issues/125?email_source=notifications&email_token=AAANFDFZKAIZGSCVUFWGZLTQ7EYR5A5CNFSM4KJ63WZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJWERLI#issuecomment-577521837, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAANFDD4GIZHYB5HSTQI3LDQ7EYR5ANCNFSM4KJ63WZQ .

[bethesque]

So, even though pact-go gets one pact per json document, for people using the native ruby who use the json formatter, they can have multiple pacts per json document. So we either need to provide an array of pact details at the top level (which for pact-go, would always have one), or set the details on each interaction.

I think I might put an array of pact details at the top level, if that works for you, and then embed the notices in the pact (basically a summarised version of the 'pact for verification' response).

{

  "pacts: [{
    "url": "",
    "consumerName": "",
    "consumerVersionTags": [...],
    "notices": [...]
  }]
}

[mefellows]

Yep that should be fine 👍

On Thu, Jan 23, 2020, 5:57 PM Beth Skurrie notifications@github.com wrote:

So, even though pact-go gets one pact per json document, for people using the native ruby who use the json formatter, they can have multiple pacts per json document. So we either need to provide an array of pact details at the top level (which for pact-go, would always have one), or set the details on each interaction.

I think I might put an array of pact details at the top level, if that works for you, and then embed the notices in the pact (basically a summarised version of the 'pact for verification' response).

{

"pacts: [{ "url": "", "consumerName": "", "consumerVersionTags": [...], "notices": [...] }] }

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/pact-foundation/pact-go/issues/125?email_source=notifications&email_token=AAANFDFUSPZWTSR34X3CTV3Q7E5XFA5CNFSM4KJ63WZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJWHCBA#issuecomment-577532164, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAANFDEEKRH2YZMZ5YZ3I3DQ7E5XFANCNFSM4KJ63WZQ .

[bethesque]

Just had the thought. When we verify multiple prod pacts, an array of string tags consumerVersionTags: [...] won't be helpful or an accurate way to identify pacts, because there will be many pacts with the tag "prod". What were you intending to use it for?

[mefellows]

True. The uniqueness is the combination of multiple things.

What I want to be able to do is structure the output of the tests so that each set of interactions is grouped by the contract and can be understood at a glance what was validated.

Let's use the following example to illustrate the point.

We call the verifier passing --all for the Mobile scoped consumer pacticipant, and --tag prod --latest for all other consumers. Let's say there are consumers A and B, and there are currently 2 versions of the Mobile consumer in prod. This should verify 4 pacts. At the moment, I don't believe I can easily construct the top-level scenario naming to look like the following:

Pact between `Provider` and `A`
   Interaction 1 scenario
      assert status code 200
      assert headers
      ...
   Interaction 2 scenario
      assert status code 404
      assert headers
      ...

Pact between `Provider` and `B`
   Interaction 1 scenario
      assert status code 200
      assert headers
      ...
   Interaction 2 scenario
      assert status code 404
      assert headers
      ...

Pact between `Provider` and `Mobile`
   Interaction 1 scenario
      assert status code 200
      assert headers
      ...
   Interaction 2 scenario
      assert status code 404
      assert headers
      ...

Pact between `Provider` and `Mobile`
   Interaction 1 scenario
      assert status code 200
      assert headers
      ...
   Interaction 2 scenario
      assert status code 404
      assert headers
      ...

Secondly, as you can see the last two scenarios need a way to disambiguate visually otherwise it's likely to be confusing - so this is where I was thinking the tags might help (but as you note, perhaps they won't). The location of the pact file from the broker is probably the most unique thing to use (so perhaps we should add that) but suffers from another annoying problem - in Go, you can't have spaces in test names, so this would format really poorly (I could extract the bits from the path to format nicer, which I think would work).

Does that make sense?

[bethesque]

I mostly understand. You need a nice, method name safe string that identifies the consumer, provider and relevant tags in a way that is also helpful for the user.

[bethesque]

Just attaching your go output image for reference.

[mefellows]

Yep, spot on!

[mefellows]

Where did we land with the inclusion of the description that includes the tags into a top-level field? I can see the notices are there.

I have a working implementation that talks to our test broker, and once we get through this I'll port over to Pact JS also.

[mefellows]

This is done.