Maven plugin generateTests goal should have an option to fail on bad contracts

[tn185070]

Is your feature request related to a problem? Please describe. We utilize contract tests to test our projects and rely on a passing build to signify that our project is working. However we have seen that passing contract tests is not necessarily a sure indicator of a project's status. If a contract definition is bad then the generateTests goal of the spring-cloud-contract-maven-plugin will just skip generating tests for that contract, and continue to generate tests for other contracts, then our tests will run the generated tests, succeed, and then we will not know that tests were not run for the cases covered by the bad contracts.

Describe the solution you'd like

We would like the spring-cloud-contract-maven-plugin to have an option to fail the maven build if a contract cannot be converted to a test.

Describe alternatives you've considered Doing maven log verification in a jenkinsfile or using the enforcer plugin.

Additional context Can provide sample project if required.

[VovkaSOL]

we need this option in gradle to)

[VovkaSOL]

I found this very bad solution, which can stop and fail build when contracs not generated, may be someone need it. parsing stdout)

def stdoutLogger = [
        onOutput: {
            if(String.valueOf(it).contains("Error Processing yaml")){
                project.ext.set("contracts_fail", true)
            }
        }
] as StandardOutputListener
gradle.taskGraph.whenReady { taskGraph ->
    project.ext.set("contracts_fail", false)
    taskGraph.allTasks.each { Task t ->
        if (t.name == 'generateContractTests') {
            t.doFirst {
                logging.addStandardOutputListener(stdoutLogger)
            }
        }
    }
}

project.gradle.taskGraph.beforeTask { task ->
        if(String.valueOf(task.name).contains("compileTestJava")){
            if(project.ext.getProperty("contracts_fail")==true){
                throw new GradleException('Generating contract tests fail=BUILD FAIL!!!')
            }
        }
}

[marwin1991]

Any progres ?

[g-fresh]

Failing the build if some of the contracts cannot be processed should be the default behavior, IMHO.

[marcingrzejszczak]

So currently we are iterating over a list of contract converters and some of them are failing (e.g. pact one would fail because JSON Wiremock stub would be parsed by the pact parser). We wouldn't want to fail the build there. OTOH maybe the problem is with how we wrote some logic in the converter in the first place.

[os-cas]

Any news on this? :)

[monsoma]

cannot imagine a situation, where I would want to test a contract, and a syntax error in my contract should not be a test failure. +1 for make this the default behaviour.

[os-cas]

I worked around this by adding a custom implementation of the org.springframework.cloud.contract.spec.ContractConverter interface (in my case for YAML contracts). In the isAccepted(File file) I delegate to the existing YamlContractConverter by spring-cloud-contract and throw an exception for invalid/ "unaccepted" contracts to let the build fail:

public class MyYamlContractVerifier implements ContractConverter<List<YamlContract>> {
    @Override
    public boolean isAccepted(File file) {
        if (!YamlContractConverter.INSTANCE.isAccepted(file)) {
            throw new IllegalStateException("Exception occurred while processing contract [" + file
                + "]. Search for the filename in the build log output for details.");
        }
        return true;
    }
    // ...
}

The custom converter is registered through the Spring factories approach by adding a spring.factories file with:

org.springframework.cloud.contract.spec.ContractConverter=\
my.java.package.MyYamlContractVerifier