arquillian / arquillian-testrunner-spock

Arquillian Spock TestRunner
Apache License 2.0
30 stars 20 forks source link

@OperateOnDeployment with @ArquillianResource not working with SPOCK 1.0 #26

Closed SchlammSpringer closed 4 years ago

SchlammSpringer commented 8 years ago

I pimped the Unroll Test from @kriegaex wich is in the issue https://github.com/arquillian/arquillian-testrunner-spock/issues/17 with just a @ArquillianResourceand now the tests won´t start anymore, is there a workaround for the SPOCK 1.0 SNAPSHOT?

thx for the help

@RunWith(ArquillianSputnik)
class ArquillianMultiDeplyomentIT extends Specification {

    def setup() { /* TODO: remove dummy method when arquillian-spock-* is fixed */ }

    @Deployment(testable = false, name = "NotTestable")
    public static WebArchive createNotTestableWarArchive() {
      final WebArchive testWar = ShrinkWrap.
          create(WebArchive.class, "ArquillianClientNotTestable.war").
          addClass(ArquillianUnrollIT.class)
      testWar
    }

    @Deployment(testable = false, name = "Testable")
    public static WebArchive createTestableWarArchive() {
      final WebArchive testWar = ShrinkWrap.
          create(WebArchive.class, "ArquillianClientStillNotTestable.war").
          addClass(ArquillianUnrollIT.class)
      testWar
    }

  @ArquillianResource
  private static URL baseUrl;

    static int counterTestableUnroll
    static int counterTestable
    static int counterNotTestable
    static int counterNotTestableUnroll

    @Unroll
    @OperateOnDeployment(value = "NotTestable")
    def "Test #test NotTestable with Unroll"() {
      when:
      counterNotTestableUnroll++

      then:
      println "++ counterNotTestableUnroll : " + counterNotTestableUnroll
      test == name

      where:
      test | name
      1    | 1
      2    | 2
      3    | 3
    }

    @OperateOnDeployment(value = "NotTestable")
    def "Test #test NotTestable without Unroll"() {
      when:
      counterNotTestable++

      then:
      println "++ counterNotTestable : " + counterNotTestable
      test == name

      where:
      test | name
      1    | 1
      2    | 2
      3    | 3
    }

    @Unroll
    @OperateOnDeployment(value = "Testable")
    def "Test #test Testable with Unroll"() {
      when:
      counterTestableUnroll++

      then:
      println "++  counterTestableUnroll : " + counterTestableUnroll
      test == name

      where:
      test | name
      1    | 1
      2    | 2
      3    | 3
    }

    @OperateOnDeployment(value = "Testable")
    def "Test #test Testable without Unroll"() {
      when:
      counterTestable++

      then:
      println "++ counterTestable : " + counterTestable
      test == name

      where:
      test | name
      1    | 1
      2    | 2
      3    | 3
    }

    def cleanupSpec() {
      println "### counterTestableUnroll : " + counterTestableUnroll
      println "### counterTestable : " + counterTestable
      println "### counterNotTestable : " + counterNotTestable
      println "### counterNotTestableUnroll : " + counterNotTestableUnroll
    }

    def cleanup() {
      println "## counterTestableUnroll : " + counterTestableUnroll
      println "## counterTestable : " + counterTestable
      println "## counterNotTestable : " + counterNotTestable
      println "## counterNotTestableUnroll : " + counterNotTestableUnroll
    }

  }```

The log looks something like this
Mär 10, 2016 3:03:00 PM org.jboss.arquillian.container.test.impl.RunModeUtils isRunAsClient
WARNUNG: The test method "de.sdvrz.okv.system.ArquillianMultiDeplyomentIT cleanup" will run on the client side - there is no running deployment yet. Please use the annotation @RunAsClient
Mär 10, 2016 3:03:00 PM org.jboss.arquillian.container.test.impl.RunModeUtils isRunAsClient
WARNUNG: The test method "de.sdvrz.okv.system.ArquillianMultiDeplyomentIT setup" will run on the client side - there is no running deployment yet. Please use the annotation @RunAsClient
Mär 10, 2016 3:03:00 PM org.jboss.arquillian.container.test.impl.RunModeUtils isRunAsClient
WARNUNG: The test method "de.sdvrz.okv.system.ArquillianMultiDeplyomentIT cleanup" will run on the client side - there is no running deployment yet. Please use the annotation @RunAsClient
Mär 10, 2016 3:03:00 PM org.jboss.arquillian.container.test.impl.RunModeUtils isRunAsClient
WARNUNG: The test method "de.sdvrz.okv.system.ArquillianMultiDeplyomentIT setup" will run on the client side - there is no running deployment yet. Please use the annotation @RunAsClient
Mär 10, 2016 3:03:00 PM org.jboss.arquillian.container.test.impl.RunModeUtils isRunAsClient
WARNUNG: The test method "de.sdvrz.okv.system.ArquillianMultiDeplyomentIT cleanup" will run on the client side - there is no running deployment yet. Please use the annotation @RunAsClient
Mär 10, 2016 3:03:00 PM org.jboss.arquillian.container.test.impl.RunModeUtils isRunAsClient
WARNUNG: The test method "de.sdvrz.okv.system.ArquillianMultiDeplyomentIT setup" will run on the client side - there is no running deployment yet. Please use the annotation @RunAsClient
Mär 10, 2016 3:03:00 PM org.jboss.arquillian.container.test.impl.RunModeUtils isRunAsClient
WARNUNG: The test method "de.sdvrz.okv.system.ArquillianMultiDeplyomentIT cleanup" will run on the client side - there is no running deployment yet. Please use the annotation @RunAsClient
Mär 10, 2016 3:03:00 PM org.jboss.arquillian.container.test.impl.RunModeUtils isRunAsClient
WARNUNG: The test method "de.sdvrz.okv.system.ArquillianMultiDeplyomentIT setup" will run on the client side - there is no running deployment yet. Please use the annotation @RunAsClient
## counterTestableUnroll : 0
## counterTestable : 0
## counterNotTestable : 0
## counterNotTestableUnroll : 0
## counterTestableUnroll : 0
## counterTestable : 0
## counterNotTestable : 0
## counterNotTestableUnroll : 0

java.lang.RuntimeException: Could not lookup value for field private static java.net.URL de.sdvrz.okv.system.ArquillianMultiDeplyomentIT.baseUrl

    at org.jboss.arquillian.test.impl.enricher.resource.ArquillianResourceTestEnricher.enrich(ArquillianResourceTestEnricher.java:71)
    at org.jboss.arquillian.test.impl.TestInstanceEnricher.enrich(TestInstanceEnricher.java:55)
    at org.jboss.arquillian.container.test.impl.ClientTestInstanceEnricher.enrich(ClientTestInstanceEnricher.java:51)
    at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
    at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)
    at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:81)
    at org.jboss.arquillian.container.test.impl.client.ContainerEventController.createContext(ContainerEventController.java:142)
    at org.jboss.arquillian.container.test.impl.client.ContainerEventController.createBeforeContext(ContainerEventController.java:124)
    at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
    at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)
    at org.jboss.arquillian.test.impl.TestContextHandler.createTestContext(TestContextHandler.java:130)
    at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
    at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)
    at org.jboss.arquillian.test.impl.TestContextHandler.createClassContext(TestContextHandler.java:92)
    at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
    at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)
    at org.jboss.arquillian.test.impl.TestContextHandler.createSuiteContext(TestContextHandler.java:73)
    at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
    at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)
    at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:145)
    at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:116)
    at org.jboss.arquillian.test.impl.EventTestRunnerAdaptor.before(EventTestRunnerAdaptor.java:108)
    at org.jboss.arquillian.spock.ArquillianInterceptor.interceptSetupMethod(ArquillianInterceptor.java:65)
    at org.spockframework.runtime.extension.AbstractMethodInterceptor.intercept(AbstractMethodInterceptor.java:28)
    at org.spockframework.runtime.extension.MethodInvocation.proceed(MethodInvocation.java:87)
    at org.jboss.arquillian.spock.ArquillianSputnik.run(ArquillianSputnik.java:145)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: java.lang.RuntimeException: Provider for type class java.net.URL returned a null value: org.jboss.arquillian.container.test.impl.enricher.resource.URLResourceProvider@4ae33a11
    at org.jboss.arquillian.test.impl.enricher.resource.ArquillianResourceTestEnricher.lookup(ArquillianResourceTestEnricher.java:133)
    at org.jboss.arquillian.test.impl.enricher.resource.ArquillianResourceTestEnricher.enrich(ArquillianResourceTestEnricher.java:67)
    ... 30 more
kriegaex commented 7 years ago

Hi @bartoszmajsak, did you have any chance to look at this one?

bartoszmajsak commented 4 years ago

Fixed in 334f5e8228fa1ba29b55ad6f1890cf5b25d42b4a

kriegaex commented 4 years ago

I copied the test into this project's examples module (running with Spock 1.3 Groovy 2.5) and ran it from there. I just adjusted the test name to your preferences (*Specification rather than *IT) and also the reference to the unroll test like this:

package org.jboss.arquillian.ftest.spock

import org.jboss.arquillian.container.test.api.Deployment
import org.jboss.arquillian.container.test.api.OperateOnDeployment
import org.jboss.arquillian.spock.ArquillianSputnik
import org.jboss.arquillian.test.api.ArquillianResource
import org.jboss.shrinkwrap.api.ShrinkWrap
import org.jboss.shrinkwrap.api.spec.WebArchive
import org.junit.runner.RunWith
import spock.lang.Specification
import spock.lang.Unroll

@RunWith(ArquillianSputnik)
class ArquillianMultiDeplyomentSpecification extends Specification {

    def setup() { /* TODO: remove dummy method when arquillian-spock-* is fixed */ }

    @Deployment(testable = false, name = "NotTestable")
    static WebArchive createNotTestableWarArchive() {
        final WebArchive testWar = ShrinkWrap.
            create(WebArchive.class, "ArquillianClientNotTestable.war").
            addClass(ArquillianUnrollSpecification.class)
        testWar
    }

    @Deployment(testable = false, name = "Testable")
    static WebArchive createTestableWarArchive() {
        final WebArchive testWar = ShrinkWrap.
            create(WebArchive.class, "ArquillianClientStillNotTestable.war").
            addClass(ArquillianUnrollSpecification.class)
        testWar
    }

    @ArquillianResource
    private static URL baseUrl

    static int counterTestableUnroll
    static int counterTestable
    static int counterNotTestable
    static int counterNotTestableUnroll

    @Unroll
    @OperateOnDeployment(value = "NotTestable")
    def "Test #test NotTestable with Unroll"() {
        when:
        counterNotTestableUnroll++

        then:
        println "++ counterNotTestableUnroll : " + counterNotTestableUnroll
        test == name

        where:
        test | name
        1    | 1
        2    | 2
        3    | 3
    }

    @OperateOnDeployment(value = "NotTestable")
    def "Test #test NotTestable without Unroll"() {
        when:
        counterNotTestable++

        then:
        println "++ counterNotTestable : " + counterNotTestable
        test == name

        where:
        test | name
        1    | 1
        2    | 2
        3    | 3
    }

    @Unroll
    @OperateOnDeployment(value = "Testable")
    def "Test #test Testable with Unroll"() {
        when:
        counterTestableUnroll++

        then:
        println "++  counterTestableUnroll : " + counterTestableUnroll
        test == name

        where:
        test | name
        1    | 1
        2    | 2
        3    | 3
    }

    @OperateOnDeployment(value = "Testable")
    def "Test #test Testable without Unroll"() {
        when:
        counterTestable++

        then:
        println "++ counterTestable : " + counterTestable
        test == name

        where:
        test | name
        1    | 1
        2    | 2
        3    | 3
    }

    def cleanupSpec() {
        println "### counterTestableUnroll : " + counterTestableUnroll
        println "### counterTestable : " + counterTestable
        println "### counterNotTestable : " + counterNotTestable
        println "### counterNotTestableUnroll : " + counterNotTestableUnroll
    }

    def cleanup() {
        println "## counterTestableUnroll : " + counterTestableUnroll
        println "## counterTestable : " + counterTestable
        println "## counterNotTestable : " + counterNotTestable
        println "## counterNotTestableUnroll : " + counterNotTestableUnroll
    }

}

On the console I still see:

java.lang.RuntimeException: Could not lookup value for field private static java.net.URL org.jboss.arquillian.ftest.spock.ArquillianMultiDeplyomentSpecification.baseUrl

BTW, if I remove this line

    def setup() { /* TODO: remove dummy method when arquillian-spock-* is fixed */ }

the error message becomes:

java.lang.IllegalArgumentException: Method '$spock_feature_0_0([class java.lang.Object, class java.lang.Object])' can't be called with parameters '[]'!

To me this looks like neither the problem in this test is fixed nor the problem that sometimes we need to manually create empty setup() methods. Please add this test to your test bed and fix the problem(s). Thank you.

More qualified feedback can be expected from my old team mate @SchlammSpringer. He was the developer in charge back then, I was just his Scrum Master and trying to help a bit.

kriegaex commented 4 years ago

I narrowed the problem down a bit. So if I add this to the existing ArquillianUnrollSpecification in your repository

--- examples/groovy-2.x/src/test/groovy/org/jboss/arquillian/ftest/spock/ArquillianUnrollSpecification.groovy   (revision 1f421e0ec78694ab6eb08309d6d9f1d9d6b907d8)
+++ examples/groovy-2.x/src/test/groovy/org/jboss/arquillian/ftest/spock/ArquillianUnrollSpecification.groovy   (date 1569649502517)
@@ -4,6 +4,7 @@
 import org.jboss.arquillian.container.test.api.OperateOnDeployment
 import org.jboss.arquillian.container.test.api.RunAsClient
 import org.jboss.arquillian.spock.ArquillianSputnik
+import org.jboss.arquillian.test.api.ArquillianResource
 import org.jboss.shrinkwrap.api.ShrinkWrap
 import org.jboss.shrinkwrap.api.spec.WebArchive
 import org.junit.runner.RunWith
@@ -13,6 +14,9 @@
 @RunWith(ArquillianSputnik)
 class ArquillianUnrollSpecification extends Specification {

+  @ArquillianResource
+  private static URL baseUrl
+
   @Deployment(testable = false, name = "NotTestable")
   static WebArchive createNotTestableWarArchive() {
     Class<?> thisClass = new Object() {}.class.enclosingClass

the test still runs successfully. But as soon as I add the empty setup() method

--- examples/groovy-2.x/src/test/groovy/org/jboss/arquillian/ftest/spock/ArquillianUnrollSpecification.groovy   (revision 1f421e0ec78694ab6eb08309d6d9f1d9d6b907d8)
+++ examples/groovy-2.x/src/test/groovy/org/jboss/arquillian/ftest/spock/ArquillianUnrollSpecification.groovy   (date 1569649502517)
@@ -4,6 +4,7 @@
 import org.jboss.arquillian.container.test.api.OperateOnDeployment
 import org.jboss.arquillian.container.test.api.RunAsClient
 import org.jboss.arquillian.spock.ArquillianSputnik
+import org.jboss.arquillian.test.api.ArquillianResource
 import org.jboss.shrinkwrap.api.ShrinkWrap
 import org.jboss.shrinkwrap.api.spec.WebArchive
 import org.junit.runner.RunWith
@@ -13,6 +14,11 @@
 @RunWith(ArquillianSputnik)
 class ArquillianUnrollSpecification extends Specification {

+  @ArquillianResource
+  private static URL baseUrl
+
+  def setup() { /* TODO: remove dummy method when arquillian-spock-* is fixed */ }
+
   @Deployment(testable = false, name = "NotTestable")
   static WebArchive createNotTestableWarArchive() {
     Class<?> thisClass = new Object() {}.class.enclosingClass

it fails with

java.lang.RuntimeException: Could not lookup value for field private static java.net.URL org.jboss.arquillian.ftest.spock.ArquillianUnrollSpecification.baseUrl

as @SchlammSpringer described.

Between his test and the original one he modified there are other semantic differences which make the test fail even without the setup() method, but I am not sure if these are intended changes or they just happened when playing with the code. I took the time to diff the two files and adjust them structurally to each other until only the semantic differences were left over:

--- examples/groovy-2.x/src/test/groovy/org/jboss/arquillian/ftest/spock/ArquillianUnrollSpecification.groovy   (revision 1f421e0ec78694ab6eb08309d6d9f1d9d6b907d8)
+++ examples/groovy-2.x/src/test/groovy/org/jboss/arquillian/ftest/spock/ArquillianUnrollSpecification.groovy   (date 1569650329337)
@@ -2,8 +2,8 @@

 import org.jboss.arquillian.container.test.api.Deployment
 import org.jboss.arquillian.container.test.api.OperateOnDeployment
-import org.jboss.arquillian.container.test.api.RunAsClient
 import org.jboss.arquillian.spock.ArquillianSputnik
+import org.jboss.arquillian.test.api.ArquillianResource
 import org.jboss.shrinkwrap.api.ShrinkWrap
 import org.jboss.shrinkwrap.api.spec.WebArchive
 import org.junit.runner.RunWith
@@ -13,6 +13,11 @@
 @RunWith(ArquillianSputnik)
 class ArquillianUnrollSpecification extends Specification {

+  @ArquillianResource
+  private static URL baseUrl
+
+//  def setup() { /* TODO: remove dummy method when arquillian-spock-* is fixed */ }
+
   @Deployment(testable = false, name = "NotTestable")
   static WebArchive createNotTestableWarArchive() {
     Class<?> thisClass = new Object() {}.class.enclosingClass
@@ -24,7 +29,7 @@
     testWar
   }

-  @Deployment(testable = true, name = "Testable")
+  @Deployment(testable = false, name = "Testable")
   static WebArchive createTestableWarArchive() {
     Class<?> thisClass = new Object() {}.class.enclosingClass
     String thisClassName = thisClass.simpleName
@@ -42,7 +47,6 @@

   @Unroll
   @OperateOnDeployment(value = "NotTestable")
-  @RunAsClient
   def "NotTestable with Unroll ##test"() {
     expect:
     ++counterNotTestableUnroll == test
@@ -51,7 +55,6 @@
   }

   @OperateOnDeployment(value = "NotTestable")
-  @RunAsClient
   def "NotTestable without Unroll"() {
     expect:
     ++counterNotTestable == test

These changes in testable = true to testable = false for the second archive and the removal of the two @RunAsClient annotations lead to the error message

java.lang.IllegalArgumentException: Method '$spock_feature_0_0([class java.lang.Object])' can't be called with parameters '[]'!

With this information it should be easy to reproduce the problem by manipulating just a few lines in the existing test.

bartoszmajsak commented 4 years ago

Thanks, I thought it's been fixed together with making your test passed. I will have a look this weekend. I know what the problem is.

kriegaex commented 4 years ago

I know what the problem is.

Thanks, @bartoszmajsak . So please re-open the ticket, thanks.

BTW, you closed a whole lot of tickets (also your own ones) at once. Did you really re-test all of the issues? Your own tickets are so terse, I don't know exactly what they mean. They were probably just reminders to yourself, so I guess you know what to do to re-test them. :-)

BTW, what does CR1 in the artifact version number mean? Something like RC1 ("release candidate 1")? I am just wondering about the CR instead of the expected RC.

bartoszmajsak commented 4 years ago

So please re-open the ticket, thanks

I think I will open separated one with your findings for clarity. This one got too broad.

BTW, you closed a whole lot of tickets (also your own ones) at once. Did you really re-test all of the issues? Your own tickets are so terse, I don't know exactly what they mean. They were probably just reminders to yourself, so I guess you know what to do to re-test them. :-)

They were simply a brain dump and an attempt to break down #18 which was quite broad ;) I think it's been covered by tests now, so I will elaborate on the descriptions.

BTW, what does CR1 in the artifact version number mean? Something like RC1 ("release candidate 1")? I am just wondering about the CR instead of the expected RC.

CR means Candidate Release and it's been used across jboss projects for years.