Open hborham opened 1 year ago
The main problem I have with this is that is adds implicit behavior. Imagine someone has previously added the environment variable and then forgotten about it. It would be very confusing to understand why the verification was not using the configured values.
The FFI provided is meant to provide the general capability for the test frameworks to use. It would be up to the particular framework to enable this (as is done with Pact Go via adding a URL source if the environment variable is set), or it would require a setting to be turned on if the functionality is to be added to the FFI verification functions.
i’ve just read your thread, you should have two separate verification tasks or task triggers, one which uses the pact url only, it can additional be provided the commit sha and branch via the webhook so the provider build can get into the correct state ( one webhook for main branch, and any deployed or released provider versions )
this happens when a consumer pact changes
a second job happens when the provider code base changes
it uses consumer version selectors, and will utilise the enable pending and include work in progress pacts.
if you provide both the pact url and also the values for a provider side change based verification ( one not triggered by a webhook ) you will see odd behaviour.
maybe the ffi verifier should give a guard rail, to only allow you to do one or the other, ( if you have a pact url, tell the user and reject any other calls with consumer version selectors / wip pacts since and enable pending )
I think we need a better way for the users to toggle between the two scenarios, and for it to be consistent across all the libraries. I don't know how the rust impl works, but the ruby shared core would let you specify both a URL and selectors, and would merge the two sets without complaint. But that's rarely a useful real world use of it, and 99% of the time, you want either selectors OR a pact URL, and putting in both suggests you have misconfigured things.
I have a canny issue for this https://pact.canny.io/feature-requests/p/support-smart-mode-toggle-where-the-verification-task-automatically-switches-bet
It's probably easy to fall into this trap like I did when utilizing the webhook template library here.
Specifically, I have a broker webhook to trigger the provider verification during an event such as
contract published that requires verification
and this webhook will set the environment variablePACT_URL
. However, it doesn't seem to do much when using pact-go@2.x.x and possibly other languages. Looking into the pact-go verifier, I would have to add code to read the environment variable when configuring the provider.I seem to recall a few PACT_* environment variables automatically read by the ruby cli tools.
Could/should
PACT_URL
be automatically configured during verification? Is this something the pact-reference project should support or is it desirable this be implemented by language implementations such as pact-go verifier?Is there a current strategy on handling environment variables automatically in the ffi tools?
See thread here.