search

pact-foundation / pact-reference

Reference implementations for the pact specifications
https://pact.io
MIT License
93 stars 46 forks source link

Verifier CLI: Support OpenTelemetry and print tracing context on interactions that fail to verify #450

Open spockz opened 5 months ago

spockz commented 5 months ago

When verifying whether a response matches the interaction the cli already prints what did and what didn't match. We are verifying numerous interactions in pipelines. We also have enabled distributed tracing (OpenTelemetry) for our applications. Since troubleshooting exactly which request failed and why it would be useful to know the trace id (traceparent) for the request that failed.

Therefore, it would be nice if the verifier would emit tracing via OpenTelemetry and print the tracing context on each failed interaction. Alternatively, if this is too large of a feature/discussion, we could also print certain response headers in case of failure. This could be used to print the traceparent header.

YOU54F commented 5 months ago

I like the sound of integrating otel with Pact, potentially as an alternative to GA for tracking, it might allow for users to point to their own data collector, or allow it go to a central source.

I have used distributed tracing with x-ray before in AWS but pretty new to otel. so would love to know more about your setup and how you invisage something looking

would you want to register with their own otel collector?

Are you testing against a deployed service rather than a local one, and have external deps that aren’t mocked out? like are you pushing these results somewhere to aggregate either in the cloud or locally?

Tangentially related, we had someone who was printing out curl statements so they could easily rerun an interaction outside of Pact

https://github.com/pact-foundation/pact-jvm/issues/1507

I quite like that and think that could certainly be capitalised on.

Anything that improves a users experience on troubleshooting failures and reducing the size of the haystack they have to investigate is right up my alley.

Could we improve the troubleshooting experience by cleaning up the logging some what so it either wasn't so exhaustive on at the point of failure, a user is armed with plenty of info of which to act upon (it currently requires a bit of searching through logs to really piece together what is going on)

mefellows commented 5 months ago

Tangentially related is this request: https://pact.canny.io/feature-requests/p/cache-provider-contract-on-pact-broker

Being able to collect and store the actual req/res payloads in the Pact Broker would be super helpful in debugging failed tests. This is different, but wanted to link it.

mefellows commented 4 months ago

There are a few elements to this we could consider:

  1. Have the verifier start the collection of oTel by adding the appropriate headers to requests (see https://www.w3.org/TR/trace-context/)
  2. Pass on and print (?) standard oTel headers received from systems
  3. Potentially send the data to an oTel collection endpoint (not mandatory, assume client has this already).
github-actions[bot] commented 4 months ago

🤖 Great news! We've labeled this issue as smartbear-supported and created a tracking ticket in PactFlow's Jira (PACT-2179). We'll keep work public and post updates here. Meanwhile, feel free to check out our docs. Thanks for your patience!