search

pact-foundation / pact-jvm

JVM version of Pact. Enables consumer driven contract testing, providing a mock service and DSL for the consumer project, and interaction playback and verification for the service provider project.
https://docs.pact.io
Apache License 2.0
1.08k stars 478 forks source link

Message provider test fails on maven-surefire-plugin 3.0.0-M4 #1108

Open zeldigas opened 4 years ago

zeldigas commented 4 years ago

After updating surefire plugin from 2.22.2 to 3.0.0-M4, pact message provider tests starts failing.

Looks like it fails to find annotated method that produce example message.

Below you can find test run (obfuscated a bit) with enabled debug logs for au.com package. The only difference is version of surefire-plugin. Not sure if it matters, but codebase is Kotlin.

2.22.2 (passes)

[INFO] Running com.example.pact.TpsCommandsProviderTest
2020-06-01 13:07:46.590 DEBUG   --- [           main] actVerificationInvocationContextProvider : provideTestTemplateInvocationContexts called
2020-06-01 13:07:46.619 DEBUG   --- [           main] actVerificationInvocationContextProvider : Verifying pacts for provider 'MYPROVIDER' and consumer 'null'
2020-06-01 13:07:46.621 DEBUG   --- [           main] actVerificationInvocationContextProvider : Pact source on test class: null
2020-06-01 13:07:46.782 DEBUG   --- [           main] actVerificationInvocationContextProvider : Pact loaders on test class: [@au.com.dius.pact.provider.junit.loader.PactBroker(scheme=${pactbroker.scheme:http}, port=${pactbroker.port:}, valueResolver=class au.com.dius.pact.core.support.expressions.SystemPropertyResolver, host=${pactbroker.host:}, consumers=[${pactbroker.consumers:}], tags=[${pactbroker.tags:latest}], authentication=@au.com.dius.pact.provider.junit.loader.PactBrokerAuth(password=${pactbroker.auth.password:}, scheme=${pactbroker.auth.scheme:legacy}, username=${pactbroker.auth.username:}, token=${pactbroker.auth.token:}))]
2020-06-01 13:07:46.816 DEBUG   --- [           main] a.c.d.p.p.junit.loader.PactBrokerLoader  : Loading pacts from pact broker for provider MYPROVIDER and tag latest
2020-06-01 13:07:46.826 DEBUG   --- [           main] a.c.d.p.p.junit.loader.PactBrokerLoader  : Authentication: basic
2020-06-01 13:07:46.848 DEBUG   --- [           main] a.c.dius.pact.core.pactbroker.HalClient  : Fetching: /
2020-06-01 13:07:49.267 DEBUG   --- [           main] a.c.dius.pact.core.pactbroker.HalClient  : Fetching: MYBROKER_URL/pacts/provider/MYPROVIDER/latest
2020-06-01 13:07:49.873 DEBUG   --- [           main] a.c.dius.pact.core.pactbroker.HalClient  : Fetching: MYBROKER_URL/pacts/provider/MYPROVIDER/consumer/tps/version/0.0.1
2020-06-01 13:07:50.688 DEBUG   --- [           main] p.j.PactVerificationStateChangeExtension : beforeEach for interaction 'INTERACTION'

Verifying a pact between tps and MYPROVIDER
  [Using Pact Broker MYBROKER_URL:]
  INTERACTION
2020-06-01 13:07:50.993 DEBUG   --- [           main] a.c.dius.pact.provider.ProviderVerifier  : projectClasspath = [file:/C:/Users/dpavlov/AppData/Local/Temp/surefire6061037475517929924/surefirebooter2462921145485977477.jar, file:/C:/Users/dpavlov/.m2/repository/org/jacoco/org.jacoco.agent/0.8.5/org.jacoco.agent-0.8.5-runtime.jar, file:/C:/Users/dpavlov/.m2/repository/net/bytebuddy/byte-buddy-agent/1.10.10/byte-buddy-agent-1.10.10.jar]
2020-06-01 13:08:03.206 DEBUG   --- [           main] a.c.dius.pact.provider.ProviderVerifier  : found class public final class com.example.pact.TpsCommandsProviderTest
2020-06-01 13:08:03.220 DEBUG   --- [           main] a.c.dius.pact.provider.ProviderVerifier  : found method [@au.com.dius.pact.provider.PactVerifyProvider("INTERACTION") @org.jetbrains.annotations.NotNull("") public final java.lang.String buildTreatmentMessage()]
2020-06-01 13:08:03.284 DEBUG   --- [           main] a.c.dius.pact.provider.ProviderVerifier  : Found methods = [public final java.lang.String com.example.pact.TpsCommandsProviderTest.buildTreatmentMessage()]
    generates a message which
.....
      has a matching body (OK)
      has matching metadata (OK)
2020-06-01 13:08:05.507 DEBUG   --- [           main] a.c.d.p.p.DefaultTestResultAccumulator   : Received test result 'au.com.dius.pact.core.pactbroker.TestResult$Ok@186dc515' for Pact MYPROVIDER-tps and INTERACTION
2020-06-01 13:08:05.515 DEBUG   --- [           main] a.c.d.p.p.DefaultTestResultAccumulator   : Number of interactions #1 and results: [au.com.dius.pact.core.pactbroker.TestResult$Ok@186dc515]
2020-06-01 13:08:05.517 DEBUG   --- [           main] a.c.d.p.p.DefaultTestResultAccumulator   : All interactions for Pact MYPROVIDER-tps have a verification result
2020-06-01 13:08:05.521  WARN   --- [           main] a.c.d.p.p.DefaultTestResultAccumulator   : Skipping publishing of verification results as it has been disabled (pact.verifier.publishResults is not 'true')
2020-06-01 13:08:05.525 DEBUG   --- [           main] p.j.PactVerificationStateChangeExtension : afterEach for interaction 'INTERACTION'

3.0.0-M4 (fails)

[INFO] Running com.example.pact.TpsCommandsProviderTest
2020-06-01 13:11:16.077 DEBUG   --- [           main] actVerificationInvocationContextProvider : provideTestTemplateInvocationContexts called
2020-06-01 13:11:16.120 DEBUG   --- [           main] actVerificationInvocationContextProvider : Verifying pacts for provider 'MYPROVIDER' and consumer 'null'
2020-06-01 13:11:16.122 DEBUG   --- [           main] actVerificationInvocationContextProvider : Pact source on test class: null
2020-06-01 13:11:16.275 DEBUG   --- [           main] actVerificationInvocationContextProvider : Pact loaders on test class: [@au.com.dius.pact.provider.junit.loader.PactBroker(scheme=${pactbroker.scheme:http}, port=${pactbroker.port:}, valueResolver=class au.com.dius.pact.core.support.expressions.SystemPropertyResolver, host=${pactbroker.host:}, consumers=[${pactbroker.consumers:}], tags=[${pactbroker.tags:latest}], authentication=@au.com.dius.pact.provider.junit.loader.PactBrokerAuth(password=${pactbroker.auth.password:}, scheme=${pactbroker.auth.scheme:legacy}, token=${pactbroker.auth.token:}, username=${pactbroker.auth.username:}))]
2020-06-01 13:11:16.316 DEBUG   --- [           main] a.c.d.p.p.junit.loader.PactBrokerLoader  : Loading pacts from pact broker for provider MYPROVIDER and tag latest
2020-06-01 13:11:16.331 DEBUG   --- [           main] a.c.d.p.p.junit.loader.PactBrokerLoader  : Authentication: basic
2020-06-01 13:11:16.363 DEBUG   --- [           main] a.c.dius.pact.core.pactbroker.HalClient  : Fetching: /
2020-06-01 13:11:18.484 DEBUG   --- [           main] a.c.dius.pact.core.pactbroker.HalClient  : Fetching: MYBROKER_URL/pacts/provider/MYPROVIDER/latest
2020-06-01 13:11:18.837 DEBUG   --- [           main] a.c.dius.pact.core.pactbroker.HalClient  : Fetching: MYBROKER_URL/pacts/provider/MYPROVIDER/consumer/tps/version/0.0.1
2020-06-01 13:11:19.863 DEBUG   --- [           main] p.j.PactVerificationStateChangeExtension : beforeEach for interaction 'INTERACTION'

Verifying a pact between tps and MYPROVIDER
  [Using Pact Broker MYBROKER_URL:]
  INTERACTION
2020-06-01 13:11:20.625 DEBUG   --- [           main] a.c.dius.pact.provider.ProviderVerifier  : projectClasspath = [file:/C:/Users/dpavlov/AppData/Local/Temp/surefire2807362591171876872/surefirebooter3628354656395821945.jar, file:/C:/Users/dpavlov/.m2/repository/org/jacoco/org.jacoco.agent/0.8.5/org.jacoco.agent-0.8.5-runtime.jar, file:/C:/Users/dpavlov/.m2/repository/net/bytebuddy/byte-buddy-agent/1.10.10/byte-buddy-agent-1.10.10.jar]
2020-06-01 13:11:21.618 DEBUG   --- [           main] a.c.dius.pact.provider.ProviderVerifier  : Found methods = []
      Verification Failed - No annotated methods were found for interaction 'INTERACTION'. You need to provide a method annotated with @PactVerifyProvider("INTERACTION") on the classpath that returns the message contents.
2020-06-01 13:11:21.736 DEBUG   --- [           main] a.c.d.p.p.DefaultTestResultAccumulator   : Received test result 'au.com.dius.pact.core.pactbroker.TestResult$Ok@63a846fe' for Pact MYPROVIDER-tps and INTERACTION
2020-06-01 13:11:21.749 DEBUG   --- [           main] a.c.d.p.p.DefaultTestResultAccumulator   : Number of interactions #1 and results: [au.com.dius.pact.core.pactbroker.TestResult$Ok@63a846fe]
2020-06-01 13:11:21.752 DEBUG   --- [           main] a.c.d.p.p.DefaultTestResultAccumulator   : All interactions for Pact MYPROVIDER-tps have a verification result
2020-06-01 13:11:21.757  WARN   --- [           main] a.c.d.p.p.DefaultTestResultAccumulator   : Skipping publishing of verification results as it has been disabled (pact.verifier.publishResults is not 'true')
2020-06-01 13:11:21.760 DEBUG   --- [           main] p.j.PactVerificationStateChangeExtension : afterEach for interaction 'INTERACTION'
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.814 s <<< FAILURE! - in com.example.pact.TpsCommandsProviderTest
[ERROR] com.example.pact.TpsCommandsProviderTest.pactVerificationTestTemplate(PactVerificationContext)[1]  Time elapsed: 2.016 s  <<< ERROR!
java.lang.IllegalArgumentException: [Ljava.lang.String;@3a52b3a2 cannot be converted to JSON
    at com.example.pact.TpsCommandsProviderTest.pactVerificationTestTemplate(TpsCommandsProviderTest.kt:33)
uglyog commented 4 years ago

I haven't been able to replicate this. My test project https://github.com/uglyog/pact-maven-amqp-test worked fine after upgrading to 3.0.0-M4.

Looking at the logs, I can see 

projectClasspath = [file:/C:/Users/dpavlov/AppData/Local/Temp/surefire2807362591171876872/surefirebooter3628354656395821945.jar, file:/C:/Users/dpavlov/.m2/repository/org/jacoco/org.jacoco.agent/0.8.5/org.jacoco.agent-0.8.5-runtime.jar, file:/C:/Users/dpavlov/.m2/repository/net/bytebuddy/byte-buddy-agent/1.10.10/byte-buddy-agent-1.10.10.jar]

From that you can see that your test code is not available on the classpath, that is why it has not found the method. Make sure you have useSystemClassLoader enabled for Surefire plugin: https://github.com/uglyog/pact-maven-amqp-test/blob/master/provider/pom.xml#L100

zeldigas commented 4 years ago

@uglyog indeed I don't have this option configured and can try to do it.

Maybe it is a separate question, but still - why do it matters? Is there any option for "lightweight" approach that does not require any class path scanning? After all junit extension has access to already created instance of test class and it'd be way simpler to just call methods on it instead of any of classpath scanning

uglyog commented 4 years ago

That was just how it was implemented.