'IncludeWipPactsSince' feature doesn't fetch pacts from feature branches

DavidHvilava commented 2 years ago

Steps I did:

pre-condition a. PactNet v4.2.1, Pactflow Version: 6205800d4 b. There is a verified contract in 'main' branch
scenario a. Create a feature branch for Consumer with changed contract. Upon pushing it calls a webhook that triggers Provider's 'verify changed pact' workflow. Expected result of that step: contract fails verification against Providers 'main' branch. b. Create a feature branch for Provider with the changes required by Consumer. Upon pushing the regular Provider workflow is triggered.

Expected result: contracts for Consumer 'main' and 'FeatureC1' branches are fetched and successful verification is provided. Actual result: only contract for Consumer 'main' branch is fetched and successful verification is provided.

My code (the value of 'IncludeWipPactsSince' is hardcoded on purpose):

[some code]
...
IPactVerifier pactVerifier = new PactVerifier(pactVerifierConfig);
IPactVerifierSource pactVerifierSource;

if (!string.IsNullOrEmpty(pactBrokerProperties.PactUrl)) {
        // For builds triggered by a 'contract content changed' webhook to verify only the changed pact.
        // The URL should be passed from the webhook to the CI job.
        pactVerifierSource = pactVerifier
            .ServiceProvider(providerName, providerUri)
            .WithUriSource(new Uri(pactBrokerProperties.PactUrl), options => {
                options
                    .TokenAuthentication(PactBrokerProperties.PactBrokerToken)
                    .PublishResults(
                        pactBrokerProperties.PublishResultsToPactBroker,
                        pactBrokerProperties.ProviderVersion,
                        results => {
                            results.ProviderBranch(pactBrokerProperties.ProviderBranch);

                            if (pactBrokerProperties.ProviderVersionTags is not null) {
                                results.ProviderTags(pactBrokerProperties.ProviderVersionTags);
                            }
                        });
            });

} else {
    // For 'normal' provider builds to fetch selected pacts for this provider
    pactVerifierSource = pactVerifier
        .ServiceProvider(providerName, providerUri)
        .WithPactBrokerSource(new Uri(PactBrokerProperties.PactBrokerBaseUrl), options => {
            options
                .TokenAuthentication(PactBrokerProperties.PactBrokerToken)
                .ConsumerVersionSelectors(pactBrokerProperties.ConsumerVersionSelectors)
                .SetPending(pactBrokerProperties.EnablePendingContracts)
                 // Hardcoded the value for visibility
                .IncludeWipPactsSince(DateTime.Parse("2021-01-01"))
                .PublishResults(
                    pactBrokerProperties.PublishResultsToPactBroker,
                    pactBrokerProperties.ProviderVersion,
                    results => {
                        results.ProviderBranch(pactBrokerProperties.ProviderBranch);

                        if (pactBrokerProperties.ProviderVersionTags is not null) {
                            results.ProviderTags(pactBrokerProperties.ProviderVersionTags);
                        }
                    });
        });
}

pactVerifierSource
    .WithProviderStateUrl(new Uri($"{PactServiceUri}/provider-states"))
    .WithFilter(filterDescription, filterProviderState)
    .WithRequestTimeout(TimeSpan.FromSeconds(requestTimeout));

pactVerifierSource.Verify();

Pact Matrix from PactFlow shows that there were no validation of the pact from Consumer FeatureC1 branch when Provider regular workflow for FeatureP1 branch were run:

Excerpt from the Provider test log:

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
The pact at https://<url is hidden>/pacts/provider/Provider1/consumer/Consumer2/pact-version/a2bdd33480e16f4983792[21](https://github.com/dhvOS/PactFlow_POC_Provider/runs/8305137202?check_suite_focus=true#step:8:22)b69cbf3d158c91d32 is being verified because the pact content belongs to the consumer version matching the following criterion:
    * latest version of Consumer2 from branch 'refs/heads/main' (3ffa5d0961a171ebffeb5a3508af71452097a199)
This pact has previously been successfully verified by Provider1. If this verification fails, it will fail the build. Read more at https://docs.pact.io/go/pending

Verifying a pact between Consumer2 and Provider1

  A invalid GET request for Date Validation without the validDateTime parameter
     Given The validDateTime parameter is not present in Consumer2
    returns a response which
      has status code 400 (OK)
      includes headers
        "Content-Type" with value "application/json; charset=utf-8" (OK)
      has a matching body (OK)

  A invalid GET request for Date Validation with invalid validDateTime parameter.
     Given The validDateTime parameter is present but has a invalid value
    returns a response which
      has status code [40](https://github.com/dhvOS/PactFlow_POC_Provider/runs/8305137202?check_suite_focus=true#step:8:41)0 (OK)
      includes headers
        "Content-Type" with value "application/json; charset=utf-8" (OK)
      has a matching body (OK)

Starting verification...
Pact verification successful

bethesque commented 2 years ago

Can you reproduce the scenario using this tool please https://github.com/pact-foundation/pact_broker/blob/master/ISSUES.md

You can copy the pacts-for-verification call with the WIP config from this file https://github.com/pact-foundation/pact_broker/blob/master/script/data/wip.rb

adamrodger commented 2 years ago

It's hard to say from looking at the code/logs provided, but at first glance it looks like you probably have some consumer version selectors defined which only match the main branch and not the feature branches. The line:

.ConsumerVersionSelectors(pactBrokerProperties.ConsumerVersionSelectors)

is important, but without knowing what those selectors are it's impossible to say whether it's configured to retrieve anything other than the main branch or not.

DavidHvilava commented 2 years ago

Hi Adam. This is a Consumer selector I have in a Provider test:

ConsumerVersionSelectors = new ConsumerVersionSelector[]
{
    new ConsumerVersionSelector() { Consumer = "Consumer2", Branch = "refs/heads/main", Latest = true }
}

But according to documentation regarding 'Work in progress pacts' we don't have to explicitly specify Consumer feature branch. Pact should take care about that.

I didn't have time to try the tool that Beth suggested. Will try it this week

DavidHvilava commented 2 years ago

Sorry for delay. I specified 'MainBranch' setting in ConsumerVersionSelector:

ConsumerVersionSelectors = new ConsumerVersionSelector[]
{
    new ConsumerVersionSelector() { Consumer = "Consumer2", MainBranch = true, Latest = true }
}

And it worked. Meaning that upon running a regular workflow for a Provider 'FeatureP1' branch that satisfies Consumer's expectations from 'main' and 'FeatureC1' branch a verification result is provided for both Consumer contracts.

I'm closing an issue. Thank you

pact-foundation / pact-net

'IncludeWipPactsSince' feature doesn't fetch pacts from feature branches #419