search

testng-team / testng

TestNG testing framework
https://testng.org
Apache License 2.0
1.99k stars 1.02k forks source link

testng ignores singleThread parameter #252

Closed nicholascus closed 4 years ago

nicholascus commented 12 years ago

Hi, I'm running Selenium tests with testNG and have following issue doing parallel test execution. Execution of test methods from one class could be halted and resumed later instead of using singleThread parameter.

I've attached small demo project which can be built with Maven https://docs.google.com/open?id=0ByfH7aqUO5g-cml5YTJ2OGRwT28

It includes suite file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd"> 
<suite verbose="2" name="Connect.Regression"> 
  <test verbose="2" name="all_tests" parallel="classes" thread-count="1"> 
    <classes>
      <class name="au.com.unick.bugz.testngparallel.Test1"/>
      <class name="au.com.unick.bugz.testngparallel.Test2"/>
      <class name="au.com.unick.bugz.testngparallel.Test3"/>
      <class name="au.com.unick.bugz.testngparallel.Test4"/>
      <class name="au.com.unick.bugz.testngparallel.Test5"/>
      <class name="au.com.unick.bugz.testngparallel.Test6"/>
    </classes>
  </test>
</suite>

and test classes like this:

@Test( singleThreaded=true )
public class Test1 extends GroupOne{
    private static Logger Log = Logger.getLogger(Test1.class.getName());

    @BeforeClass
    public void test1_beforeClass(){
        Log.info(testName+": test1_beforeClass");
    }

    @Test
    public void test1_test1(){
        Log.info(testName+": test1_1");
    }

    @Test(dependsOnMethods={"test1_test1"})
    public void test1_test2(){
        Log.info(testName+": test1_2");
    }
}

executing that suite file you can see following output:

 T E S T S
-------------------------------------------------------
Running TestSuite
...
... TestNG 6.7 by C?dric Beust (cedric@beust.com)
...

[TestRunner] Starting executor for test all_tests with time out:2147483647 milliseconds.
[pool-1-thread-1][BaseTest] 27 Jul 2012 13:34:12,995  INFO  Test1: baseTest_beforeClass
[pool-1-thread-1][GroupOne] 27 Jul 2012 13:34:13,002  INFO  Test1: groupOne_beforeClass
[pool-1-thread-1][Test1] 27 Jul 2012 13:34:13,002  INFO  Test1: test1_beforeClass
[pool-1-thread-1][Test1] 27 Jul 2012 13:34:13,005  INFO  Test1: test1_1
[pool-1-thread-1][BaseTest] 27 Jul 2012 13:34:13,011  INFO  Test2: baseTest_beforeClass
[pool-1-thread-1][GroupOne] 27 Jul 2012 13:34:13,012  INFO  Test2: groupOne_beforeClass
[pool-1-thread-1][Test2] 27 Jul 2012 13:34:13,012  INFO  Test2: test2_beforeClass
[pool-1-thread-1][Test2] 27 Jul 2012 13:34:13,013  INFO  Test2: test2_1
[pool-1-thread-1][BaseTest] 27 Jul 2012 13:34:13,018  INFO  Test3: baseTest_beforeClass
[pool-1-thread-1][GroupTwo] 27 Jul 2012 13:34:13,019  INFO  Test3: groupTwo_beforeClass
[pool-1-thread-1][Test3] 27 Jul 2012 13:34:13,019  INFO  Test3: test3_beforeClass
[pool-1-thread-1][Test3] 27 Jul 2012 13:34:13,020  INFO  Test3: test3_1
[pool-1-thread-1][BaseTest] 27 Jul 2012 13:34:13,025  INFO  Test4: baseTest_beforeClass
[pool-1-thread-1][GroupTwo] 27 Jul 2012 13:34:13,026  INFO  Test4: groupTwo_beforeClass
[pool-1-thread-1][Test4] 27 Jul 2012 13:34:13,026  INFO  Test4: test4_beforeClass
[pool-1-thread-1][Test4] 27 Jul 2012 13:34:13,026  INFO  Test4: test4_1
[pool-1-thread-1][BaseTest] 27 Jul 2012 13:34:13,032  INFO  Test5: baseTest_beforeClass
[pool-1-thread-1][Test5] 27 Jul 2012 13:34:13,033  INFO  Test5: test5_beforeClass
[pool-1-thread-1][Test5] 27 Jul 2012 13:34:13,033  INFO  Test5: test5_1
[pool-1-thread-1][BaseTest] 27 Jul 2012 13:34:13,038  INFO  Test6: baseTest_beforeClass
[pool-1-thread-1][Test6] 27 Jul 2012 13:34:13,039  INFO  Test6: test6_beforeClass
[pool-1-thread-1][Test6] 27 Jul 2012 13:34:13,039  INFO  Test6: test6_1
[pool-1-thread-1][Test1] 27 Jul 2012 13:34:13,041  INFO  Test1: test1_2
[pool-1-thread-1][BaseTest] 27 Jul 2012 13:34:13,041  INFO  Test1: baseTest_afterClass
[pool-1-thread-1][Test2] 27 Jul 2012 13:34:13,042  INFO  Test2: test2_2
[pool-1-thread-1][BaseTest] 27 Jul 2012 13:34:13,042  INFO  Test2: baseTest_afterClass
[pool-1-thread-1][Test3] 27 Jul 2012 13:34:13,043  INFO  Test3: test3_2
[pool-1-thread-1][BaseTest] 27 Jul 2012 13:34:13,043  INFO  Test3: baseTest_afterClass
[pool-1-thread-1][Test4] 27 Jul 2012 13:34:13,043  INFO  Test4: test4_2
[pool-1-thread-1][BaseTest] 27 Jul 2012 13:34:13,044  INFO  Test4: baseTest_afterClass
[pool-1-thread-1][Test5] 27 Jul 2012 13:34:13,044  INFO  Test5: test5_2
[pool-1-thread-1][BaseTest] 27 Jul 2012 13:34:13,045  INFO  Test5: baseTest_afterClass
[pool-1-thread-1][Test6] 27 Jul 2012 13:34:13,045  INFO  Test6: test6_1
[pool-1-thread-1][BaseTest] 27 Jul 2012 13:34:13,046  INFO  Test6: baseTest_afterClass

You can see that Test1 class execution was interrupted after first testing method, then resumed after processing of some other tests.

It's possible to see the same issue with other thread-count limits, you will just have less illustrative test output.

nicholascus commented 12 years ago

testng beta logs:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Forking command line: cmd.exe /X /C ""C:\Program Files (x86)\Java\jdk1.6.0_27\jre\bin\java" -jar D:\Eclipse_Workspace\test-ng-test\target\surefire\surefirebooter6112431790442324144.jar D:\Eclipse_Workspace\test-ng-test\target\surefire\surefire1033648408041040366tmp D:\Eclipse_Workspace\test-ng-test\target\surefire\surefire_07260632390826980493tmp"
Running TestSuite
...
... TestNG 6.6beta 20120812_2350 by Cédric Beust (cedric@beust.com)
...

[TestRunner] Running the tests in 'all_tests' with parallel mode:false
[RunInfo] Adding method selector: org.testng.internal.XmlMethodSelector@1c39a2d priority: 10
[TestClass] Creating TestClass for [ClassImpl au.com.unick.bugz.testngparallel.Test1]
[TestClass] Adding method Test1.test1_test1()[pri:0, instance:null] on TestClass class au.com.unick.bugz.testngparallel.Test1
[TestClass] Adding method Test1.test1_test2()[pri:0, instance:null] on TestClass class au.com.unick.bugz.testngparallel.Test1
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.test1_beforeClass()
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.baseTest_beforeClass()
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.groupOne_beforeClass()
[MethodInheritance] GroupOne.groupOne_beforeClass()[pri:0, instance:null] DEPENDS ON BaseTest.baseTest_beforeClass()[pri:0, instance:null]
[MethodInheritance] Test1.test1_beforeClass()[pri:0, instance:null] DEPENDS ON BaseTest.baseTest_beforeClass()[pri:0, instance:null]
[MethodInheritance] Test1.test1_beforeClass()[pri:0, instance:null] DEPENDS ON GroupOne.groupOne_beforeClass()[pri:0, instance:null]
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.baseTest_afterClass()
[TestClass] Creating TestClass for [ClassImpl au.com.unick.bugz.testngparallel.Test3]
[TestClass] Adding method Test3.test3_test2()[pri:0, instance:null] on TestClass class au.com.unick.bugz.testngparallel.Test3
[TestClass] Adding method Test3.test3_test1()[pri:0, instance:null] on TestClass class au.com.unick.bugz.testngparallel.Test3
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.test3_beforeClass()
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.groupTwo_beforeClass()
[MethodInheritance] GroupTwo.groupTwo_beforeClass()[pri:0, instance:null] DEPENDS ON BaseTest.baseTest_beforeClass()[pri:0, instance:null]
[MethodInheritance] Test3.test3_beforeClass()[pri:0, instance:null] DEPENDS ON BaseTest.baseTest_beforeClass()[pri:0, instance:null]
[MethodInheritance] Test3.test3_beforeClass()[pri:0, instance:null] DEPENDS ON GroupTwo.groupTwo_beforeClass()[pri:0, instance:null]
[TestClass] Creating TestClass for [ClassImpl au.com.unick.bugz.testngparallel.Test4]
[TestClass] Adding method Test4.test4_test2()[pri:0, instance:null] on TestClass class au.com.unick.bugz.testngparallel.Test4
[TestClass] Adding method Test4.test4_test1()[pri:0, instance:null] on TestClass class au.com.unick.bugz.testngparallel.Test4
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.test4_beforeClass()
[MethodInheritance] GroupTwo.groupTwo_beforeClass()[pri:0, instance:null] DEPENDS ON BaseTest.baseTest_beforeClass()[pri:0, instance:null]
[MethodInheritance] Test4.test4_beforeClass()[pri:0, instance:null] DEPENDS ON BaseTest.baseTest_beforeClass()[pri:0, instance:null]
[MethodInheritance] Test4.test4_beforeClass()[pri:0, instance:null] DEPENDS ON GroupTwo.groupTwo_beforeClass()[pri:0, instance:null]
[TestClass] Creating TestClass for [ClassImpl au.com.unick.bugz.testngparallel.Test6]
[TestClass] Adding method Test6.test6_test2()[pri:0, instance:null] on TestClass class au.com.unick.bugz.testngparallel.Test6
[TestClass] Adding method Test6.test6_test1()[pri:0, instance:null] on TestClass class au.com.unick.bugz.testngparallel.Test6
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.test6_beforeClass()
[MethodInheritance] Test6.test6_beforeClass()[pri:0, instance:null] DEPENDS ON BaseTest.baseTest_beforeClass()[pri:0, instance:null]
[TestClass] Creating TestClass for [ClassImpl au.com.unick.bugz.testngparallel.Test2]
[TestClass] Adding method Test2.test1_test2()[pri:0, instance:null] on TestClass class au.com.unick.bugz.testngparallel.Test2
[TestClass] Adding method Test2.test2_test1()[pri:0, instance:null] on TestClass class au.com.unick.bugz.testngparallel.Test2
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.test2_beforeClass()
[MethodInheritance] GroupOne.groupOne_beforeClass()[pri:0, instance:null] DEPENDS ON BaseTest.baseTest_beforeClass()[pri:0, instance:null]
[MethodInheritance] Test2.test2_beforeClass()[pri:0, instance:null] DEPENDS ON BaseTest.baseTest_beforeClass()[pri:0, instance:null]
[MethodInheritance] Test2.test2_beforeClass()[pri:0, instance:null] DEPENDS ON GroupOne.groupOne_beforeClass()[pri:0, instance:null]
[TestClass] Creating TestClass for [ClassImpl au.com.unick.bugz.testngparallel.Test5]
[TestClass] Adding method Test5.test5_test2()[pri:0, instance:null] on TestClass class au.com.unick.bugz.testngparallel.Test5
[TestClass] Adding method Test5.test5_test1()[pri:0, instance:null] on TestClass class au.com.unick.bugz.testngparallel.Test5
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.test5_beforeClass()
[MethodInheritance] Test5.test5_beforeClass()[pri:0, instance:null] DEPENDS ON BaseTest.baseTest_beforeClass()[pri:0, instance:null]
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.test1_test1()
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.test1_test2()
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.test3_test2()
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.test3_test1()
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.test4_test2()
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.test4_test1()
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.test6_test2()
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.test6_test1()
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.test5_test2()
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.test5_test1()
[XmlMethodSelector] Including method au.com.unick.bugz.testngparallel.test2_test1()
[SuiteRunner] Created 1 TestRunners
[TestRunner] Running test all_tests on 6  classes,  included groups:[] excluded groups:[]
===== Test class
au.com.unick.bugz.testngparallel.Test1
  @BeforeClass BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c]
  @BeforeClass GroupOne.groupOne_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c]
  @BeforeClass Test1.test1_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c]
    @Test Test1.test1_test1()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c]
    @Test Test1.test1_test2()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c]
  @AfterClass BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c]
======
===== Test class
au.com.unick.bugz.testngparallel.Test3
  @BeforeClass BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02]
  @BeforeClass GroupTwo.groupTwo_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02]
  @BeforeClass Test3.test3_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02]
    @Test Test3.test3_test2()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02]
    @Test Test3.test3_test1()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02]
  @AfterClass BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02]
======
===== Test class
au.com.unick.bugz.testngparallel.Test4
  @BeforeClass BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993]
  @BeforeClass GroupTwo.groupTwo_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993]
  @BeforeClass Test4.test4_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993]
    @Test Test4.test4_test2()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993]
    @Test Test4.test4_test1()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993]
  @AfterClass BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993]
======
===== Test class
au.com.unick.bugz.testngparallel.Test6
  @BeforeClass BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test6@b8f82d]
  @BeforeClass Test6.test6_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test6@b8f82d]
    @Test Test6.test6_test2()[pri:0, instance:au.com.unick.bugz.testngparallel.Test6@b8f82d]
    @Test Test6.test6_test1()[pri:0, instance:au.com.unick.bugz.testngparallel.Test6@b8f82d]
  @AfterClass BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test6@b8f82d]
======
===== Test class
au.com.unick.bugz.testngparallel.Test5
  @BeforeClass BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test5@1b1aa65]
  @BeforeClass Test5.test5_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test5@1b1aa65]
    @Test Test5.test5_test2()[pri:0, instance:au.com.unick.bugz.testngparallel.Test5@1b1aa65]
    @Test Test5.test5_test1()[pri:0, instance:au.com.unick.bugz.testngparallel.Test5@1b1aa65]
  @AfterClass BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test5@1b1aa65]
======
===== Test class
au.com.unick.bugz.testngparallel.Test2
  @BeforeClass BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f]
  @BeforeClass GroupOne.groupOne_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f]
  @BeforeClass Test2.test2_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f]
    @Test Test2.test1_test2()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f]
    @Test Test2.test2_test1()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f]
  @AfterClass BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f]
======
[TestRunner] Starting executor for test all_tests with time out:2147483647 milliseconds.
[Invoker 13783459] Keeping method BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c] for class [TestClass class au.com.unick.bugz.testngparallel.Test1]
[Invoker 13783459] Keeping method GroupOne.groupOne_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c] for class [TestClass class au.com.unick.bugz.testngparallel.Test1]
[Invoker 13783459] Keeping method Test1.test1_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c] for class [TestClass class au.com.unick.bugz.testngparallel.Test1]
[Invoker 13783459] Invoking @BeforeClass BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c]
[pool-1-thread-1][BaseTest] 27 Jul 2012 16:46:11,804  INFO  Test1: baseTest_beforeClass
[Invoker 13783459] Invoking @BeforeClass GroupOne.groupOne_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c]
[pool-1-thread-1][GroupOne] 27 Jul 2012 16:46:11,807  INFO  Test1: groupOne_beforeClass
[Invoker 13783459] Invoking @BeforeClass Test1.test1_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c]
[pool-1-thread-1][Test1] 27 Jul 2012 16:46:11,812  INFO  Test1: test1_beforeClass
[Invoker 13783459] Invoking au.com.unick.bugz.testngparallel.Test1.test1_test1
[pool-1-thread-1][Test1] 27 Jul 2012 16:46:11,814  INFO  Test1: test1_1
[Invoker 13783459] Keeping method BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f] for class [TestClass class au.com.unick.bugz.testngparallel.Test2]
[Invoker 13783459] Keeping method GroupOne.groupOne_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f] for class [TestClass class au.com.unick.bugz.testngparallel.Test2]
[Invoker 13783459] Keeping method Test2.test2_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f] for class [TestClass class au.com.unick.bugz.testngparallel.Test2]
[Invoker 13783459] Invoking @BeforeClass BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f]
[pool-1-thread-1][BaseTest] 27 Jul 2012 16:46:11,822  INFO  Test2: baseTest_beforeClass
[Invoker 13783459] Invoking @BeforeClass GroupOne.groupOne_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f]
[pool-1-thread-1][GroupOne] 27 Jul 2012 16:46:11,822  INFO  Test2: groupOne_beforeClass
[Invoker 13783459] Invoking @BeforeClass Test2.test2_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f]
[pool-1-thread-1][Test2] 27 Jul 2012 16:46:11,822  INFO  Test2: test2_beforeClass
[Invoker 13783459] Invoking au.com.unick.bugz.testngparallel.Test2.test2_test1
[pool-1-thread-1][Test2] 27 Jul 2012 16:46:11,824  INFO  Test2: test2_1
[Invoker 13783459] Keeping method BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02] for class [TestClass class au.com.unick.bugz.testngparallel.Test3]
[Invoker 13783459] Keeping method GroupTwo.groupTwo_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02] for class [TestClass class au.com.unick.bugz.testngparallel.Test3]
[Invoker 13783459] Keeping method Test3.test3_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02] for class [TestClass class au.com.unick.bugz.testngparallel.Test3]
[Invoker 13783459] Invoking @BeforeClass BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02]
[pool-1-thread-1][BaseTest] 27 Jul 2012 16:46:11,829  INFO  Test3: baseTest_beforeClass
[Invoker 13783459] Invoking @BeforeClass GroupTwo.groupTwo_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02]
[pool-1-thread-1][GroupTwo] 27 Jul 2012 16:46:11,829  INFO  Test3: groupTwo_beforeClass
[Invoker 13783459] Invoking @BeforeClass Test3.test3_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02]
[pool-1-thread-1][Test3] 27 Jul 2012 16:46:11,832  INFO  Test3: test3_beforeClass
[Invoker 13783459] Invoking au.com.unick.bugz.testngparallel.Test3.test3_test1
[pool-1-thread-1][Test3] 27 Jul 2012 16:46:11,832  INFO  Test3: test3_1
[Invoker 13783459] Keeping method BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993] for class [TestClass class au.com.unick.bugz.testngparallel.Test4]
[Invoker 13783459] Keeping method GroupTwo.groupTwo_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993] for class [TestClass class au.com.unick.bugz.testngparallel.Test4]
[Invoker 13783459] Keeping method Test4.test4_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993] for class [TestClass class au.com.unick.bugz.testngparallel.Test4]
[Invoker 13783459] Invoking @BeforeClass BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993]
[pool-1-thread-1][BaseTest] 27 Jul 2012 16:46:11,837  INFO  Test4: baseTest_beforeClass
[Invoker 13783459] Invoking @BeforeClass GroupTwo.groupTwo_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993]
[pool-1-thread-1][GroupTwo] 27 Jul 2012 16:46:11,837  INFO  Test4: groupTwo_beforeClass
[Invoker 13783459] Invoking @BeforeClass Test4.test4_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993]
[pool-1-thread-1][Test4] 27 Jul 2012 16:46:11,839  INFO  Test4: test4_beforeClass
[Invoker 13783459] Invoking au.com.unick.bugz.testngparallel.Test4.test4_test1
[pool-1-thread-1][Test4] 27 Jul 2012 16:46:11,839  INFO  Test4: test4_1
[Invoker 13783459] Keeping method BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test5@1b1aa65] for class [TestClass class au.com.unick.bugz.testngparallel.Test5]
[Invoker 13783459] Keeping method Test5.test5_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test5@1b1aa65] for class [TestClass class au.com.unick.bugz.testngparallel.Test5]
[Invoker 13783459] Invoking @BeforeClass BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test5@1b1aa65]
[pool-1-thread-1][BaseTest] 27 Jul 2012 16:46:11,844  INFO  Test5: baseTest_beforeClass
[Invoker 13783459] Invoking @BeforeClass Test5.test5_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test5@1b1aa65]
[pool-1-thread-1][Test5] 27 Jul 2012 16:46:11,844  INFO  Test5: test5_beforeClass
[Invoker 13783459] Invoking au.com.unick.bugz.testngparallel.Test5.test5_test1
[pool-1-thread-1][Test5] 27 Jul 2012 16:46:11,847  INFO  Test5: test5_1
[Invoker 13783459] Keeping method BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test6@b8f82d] for class [TestClass class au.com.unick.bugz.testngparallel.Test6]
[Invoker 13783459] Keeping method Test6.test6_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test6@b8f82d] for class [TestClass class au.com.unick.bugz.testngparallel.Test6]
[Invoker 13783459] Invoking @BeforeClass BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test6@b8f82d]
[pool-1-thread-1][BaseTest] 27 Jul 2012 16:46:11,852  INFO  Test6: baseTest_beforeClass
[Invoker 13783459] Invoking @BeforeClass Test6.test6_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test6@b8f82d]
[pool-1-thread-1][Test6] 27 Jul 2012 16:46:11,854  INFO  Test6: test6_beforeClass
[Invoker 13783459] Invoking au.com.unick.bugz.testngparallel.Test6.test6_test1
[pool-1-thread-1][Test6] 27 Jul 2012 16:46:11,854  INFO  Test6: test6_1
[Invoker 13783459] Invoking au.com.unick.bugz.testngparallel.Test1.test1_test2
[pool-1-thread-1][Test1] 27 Jul 2012 16:46:11,854  INFO  Test1: test1_2
[Invoker 13783459] Keeping method BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c] for class [TestClass class au.com.unick.bugz.testngparallel.Test1]
[Invoker 13783459] Invoking @AfterClass BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c]
[pool-1-thread-1][BaseTest] 27 Jul 2012 16:46:11,857  INFO  Test1: baseTest_afterClass
[Invoker 13783459] Invoking au.com.unick.bugz.testngparallel.Test2.test1_test2
[pool-1-thread-1][Test2] 27 Jul 2012 16:46:11,857  INFO  Test2: test2_2
[Invoker 13783459] Keeping method BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f] for class [TestClass class au.com.unick.bugz.testngparallel.Test2]
[Invoker 13783459] Invoking @AfterClass BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f]
[pool-1-thread-1][BaseTest] 27 Jul 2012 16:46:11,857  INFO  Test2: baseTest_afterClass
[Invoker 13783459] Invoking au.com.unick.bugz.testngparallel.Test3.test3_test2
[pool-1-thread-1][Test3] 27 Jul 2012 16:46:11,859  INFO  Test3: test3_2
[Invoker 13783459] Keeping method BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02] for class [TestClass class au.com.unick.bugz.testngparallel.Test3]
[Invoker 13783459] Invoking @AfterClass BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02]
[pool-1-thread-1][BaseTest] 27 Jul 2012 16:46:11,859  INFO  Test3: baseTest_afterClass
[Invoker 13783459] Invoking au.com.unick.bugz.testngparallel.Test4.test4_test2
[pool-1-thread-1][Test4] 27 Jul 2012 16:46:11,859  INFO  Test4: test4_2
[Invoker 13783459] Keeping method BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993] for class [TestClass class au.com.unick.bugz.testngparallel.Test4]
[Invoker 13783459] Invoking @AfterClass BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993]
[pool-1-thread-1][BaseTest] 27 Jul 2012 16:46:11,859  INFO  Test4: baseTest_afterClass
[Invoker 13783459] Invoking au.com.unick.bugz.testngparallel.Test5.test5_test2
[pool-1-thread-1][Test5] 27 Jul 2012 16:46:11,862  INFO  Test5: test5_2
[Invoker 13783459] Keeping method BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test5@1b1aa65] for class [TestClass class au.com.unick.bugz.testngparallel.Test5]
[Invoker 13783459] Invoking @AfterClass BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test5@1b1aa65]
[pool-1-thread-1][BaseTest] 27 Jul 2012 16:46:11,862  INFO  Test5: baseTest_afterClass
[Invoker 13783459] Invoking au.com.unick.bugz.testngparallel.Test6.test6_test2
[pool-1-thread-1][Test6] 27 Jul 2012 16:46:11,862  INFO  Test6: test6_1
[Invoker 13783459] Keeping method BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test6@b8f82d] for class [TestClass class au.com.unick.bugz.testngparallel.Test6]
[Invoker 13783459] Invoking @AfterClass BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test6@b8f82d]
[pool-1-thread-1][BaseTest] 27 Jul 2012 16:46:11,862  INFO  Test6: baseTest_afterClass
===== Invoked methods
  BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c] 28061596
  GroupOne.groupOne_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c] 28061596
  Test1.test1_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c] 28061596
    Test1.test1_test1()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c] 28061596
  BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f] 19327071
  GroupOne.groupOne_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f] 19327071
  Test2.test2_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f] 19327071
    Test2.test2_test1()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f] 19327071
  BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02] 2263554
  GroupTwo.groupTwo_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02] 2263554
  Test3.test3_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02] 2263554
    Test3.test3_test1()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02] 2263554
  BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993] 3705235
  GroupTwo.groupTwo_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993] 3705235
  Test4.test4_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993] 3705235
    Test4.test4_test1()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993] 3705235
  BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test5@1b1aa65] 28420709
  Test5.test5_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test5@1b1aa65] 28420709
    Test5.test5_test1()[pri:0, instance:au.com.unick.bugz.testngparallel.Test5@1b1aa65] 28420709
  BaseTest.baseTest_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test6@b8f82d] 12122157
  Test6.test6_beforeClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test6@b8f82d] 12122157
    Test6.test6_test1()[pri:0, instance:au.com.unick.bugz.testngparallel.Test6@b8f82d] 12122157
    Test1.test1_test2()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c] 28061596
  BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test1@1ac2f9c] 28061596
    Test2.test1_test2()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f] 19327071
  BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test2@126e85f] 19327071
    Test3.test3_test2()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02] 2263554
  BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test3@228a02] 2263554
    Test4.test4_test2()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993] 3705235
  BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test4@388993] 3705235
    Test5.test5_test2()[pri:0, instance:au.com.unick.bugz.testngparallel.Test5@1b1aa65] 28420709
  BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test5@1b1aa65] 28420709
    Test6.test6_test2()[pri:0, instance:au.com.unick.bugz.testngparallel.Test6@b8f82d] 12122157
  BaseTest.baseTest_afterClass()[pri:0, instance:au.com.unick.bugz.testngparallel.Test6@b8f82d] 12122157
krmahadevan commented 7 years ago

test-ng-test.zip

Adding the project as a zip, just incase the google drive link stops working.

dasAnderl commented 6 years ago

i have the some problem even if tests are executed non parallel. dependson is considered correctly, but methods from different classes get mixed. it seems to be quite random. strange is that in the testcontext (access through listener onStart method) the methods are ordered correctly. i dont know where this order gets mixed up see the following test execution (expected would be that same prefixes are run together as the reside version 7.0.0-SNAPSHOT and latest 6 in the same class e.g. all GEOXX should run together):

start GEO01_post_def_list end GEO01_post_def_list start GEO_NEG01_post_def_list failed GEO_NEG01_post_def_list start GEO_NEG01_post_def_list failed GEO_NEG01_post_def_list start RSA01_GetSpeedAlertConfig end RSA01_GetSpeedAlertConfig start RSA01_GetSpeedAlertConfig end RSA01_GetSpeedAlertConfig start RSA02_SaveSpeedAlertDefList end RSA02_SaveSpeedAlertDefList start RSA02_SaveSpeedAlertDefList failed RSA02_SaveSpeedAlertDefList start RTS01_PostVehicleTripData end RTS01_PostVehicleTripData start RTS02_GetTripDataListLongTerm end RTS02_GetTripDataListLongTerm start RTS03_GetTripDataListShortTerm end RTS03_GetTripDataListShortTerm start RTS04_GetTripDataListCyclic failed RTS04_GetTripDataListCyclic start RTS05_GetNewestTripDataLongTerm end RTS05_GetNewestTripDataLongTerm start RTS06_GetNewestTripDataShortTerm end RTS06_GetNewestTripDataShortTerm start RTS07_GetNewestTripDataCyclic failed RTS07_GetNewestTripDataCyclic start RTS08_GetAggregatedTripDataLongTerm end RTS08_GetAggregatedTripDataLongTerm start RTS09_GetAggregatedTripDataShortTerm end RTS09_GetAggregatedTripDataShortTerm start RTS10_GetAggregatedTripDataCyclic failed RTS10_GetAggregatedTripDataCyclic start RTS11_DeleteTripDataLongTerm end RTS11_DeleteTripDataLongTerm start RTS12_DeleteTripDataShortTerm end RTS12_DeleteTripDataShortTerm start RTS13_DeleteTripDataCyclic end RTS13_DeleteTripDataCyclic start RTS14_PostVehicleTripData end RTS14_PostVehicleTripData start RTS15_DeleteLongTermTripDataById end RTS15_DeleteLongTermTripDataById start RTS16_DeleteShortTermTripDataById end RTS16_DeleteShortTermTripDataById start RTS17_DeleteCyclicTripDataById failed RTS17_DeleteCyclicTripDataById start RTS18_PostVehicleTripData end RTS18_PostVehicleTripData start RTS19_DeleteLongTermTripDataByMileage end RTS19_DeleteLongTermTripDataByMileage start RTS20_DeleteShortTermTripDataByMileage end RTS20_DeleteShortTermTripDataByMileage start RTS21_DeleteCyclicTripDataByMileage failed RTS21_DeleteCyclicTripDataByMileage skipped GEO01_post_def_list start GEO02_verify_status_is_pending end GEO02_verify_status_is_pending skipped GEO_NEG02_verify_status_is_pending skipped GEO_NEG02_verify_status_is_pending start RSA03_GetDefListStatus end RSA03_GetDefListStatus skipped RSA03_GetDefListStatus skipped GEO02_verify_status_is_pending start GEO03_acknowledge_joblist end GEO03_acknowledge_joblist skipped GEO_NEG03_acknowledge_joblist skipped GEO_NEG03_acknowledge_joblist start RSA04_GetAndAckJob end RSA04_GetAndAckJob skipped RSA04_GetAndAckJob skipped GEO03_acknowledge_joblist start GEO04_post_JobProcessingCallback_noError end GEO04_post_JobProcessingCallback_noError skipped GEO_NEG04_post_JobProcessingCallback_error skipped GEO_NEG04_post_JobProcessingCallback_error start RSA05_GetJobProcessingCallback end RSA05_GetJobProcessingCallback skipped RSA05_GetJobProcessingCallback skipped GEO04_post_JobProcessingCallback_noError start GEO05_verify_status_is_acknowledged end GEO05_verify_status_is_acknowledged skipped GEO_NEG05_verify_status_is_error skipped GEO_NEG05_verify_status_is_error start RSA06_GetDefListStatus end RSA06_GetDefListStatus skipped RSA06_GetDefListStatus skipped GEO05_verify_status_is_acknowledged start GEO06_post_violations end GEO06_post_violations skipped GEO_NEG06_verify_history_contains_error skipped GEO_NEG06_verify_history_contains_error start RSA07_GetSpeedAlertDefinitionList end RSA07_GetSpeedAlertDefinitionList skipped RSA07_GetSpeedAlertDefinitionList skipped GEO06_post_violations start GEO07_verify_history_contains_activate end GEO07_verify_history_contains_activate start RSA08_PostViolation end RSA08_PostViolation skipped RSA08_PostViolation skipped GEO07_verify_history_contains_activate start GEO08_get_and_verify_alerts end GEO08_get_and_verify_alerts start RSA09_GetSpeedAlertHistory end RSA09_GetSpeedAlertHistory skipped RSA09_GetSpeedAlertHistory skipped GEO08_get_and_verify_alerts start GEO09_get_configuration end GEO09_get_configuration start RSA10_GetSpeedAlerts end RSA10_GetSpeedAlerts skipped RSA10_GetSpeedAlerts skipped GEO09_get_configuration start GEO10_delete_one_alert end GEO10_delete_one_alert start RSA11_DeleteSpeedAlertList end RSA11_DeleteSpeedAlertList skipped RSA11_DeleteSpeedAlertList skipped GEO10_delete_one_alert start GEO11_delete_all_alerts end GEO11_delete_all_alerts start RSA12_DeleteRequestHistoryEntriesList end RSA12_DeleteRequestHistoryEntriesList skipped RSA12_DeleteRequestHistoryEntriesList skipped GEO11_delete_all_alerts start GEO12_delete_one_history_entry end GEO12_delete_one_history_entry skipped GEO12_delete_one_history_entry start GEO13_delete_history end GEO13_delete_history skipped GEO13_delete_history

dasAnderl commented 6 years ago

using class annotation @Test(suiteName = "Dwap") on each test fixes the issue

adrian-herscu-ge commented 5 years ago

using class annotation @test(suiteName = "Dwap") on each test fixes the issue

"Dwap" ?? What's this?

krmahadevan commented 4 years ago

This is working fine when involving only independent methods (no usage of dependsOnMethods and dependsOnGroups)

The attached project can be used for cross checking.

When either dependsOnMethods (or) dependsOnGroups is used, this assurance is broken.

TestNG did introduce a feature called thread affinity wherein downstream methods will run on the same thread as the upstream methods (only when using dependsOnMethods and with a single dependency), but even this feature is limited in capabilities.

In short, one shouldn't be relying on thread affinity and expect it to be honored when there are test dependencies involved.

issue_252.zip