testng-team / testng

TestNG testing framework
https://testng.org
Apache License 2.0
1.98k stars 1.02k forks source link

Combined in the same testNG class priorities and dependOnMethods issue #2052

Closed sdrss closed 5 years ago

sdrss commented 5 years ago

In a testng class with combined priorities and dependsOnMethods some @tests are not running. I can't be sure about the testNG version that caused this issue. I am currently running with 6.14.3

See code sample below ;

Works fine ... if you have only 2 @tests after the one with the priority. If you have more than 2 , then only the @test with the priority is executed and the rests are gone ... meaning not executed and not reported anywhere. Results are showing (for the code sample below ) :

[RemoteTestNG] detected TestNG version 6.14.3 PASSED: test1 Default test Tests run: 1, Failures: 0, Skips: 0

In the same code sample if you comment out the test4 , then all works as expected.

TestNG Version

6.14.3

Expected behavior

All @Tests should run

Actual behavior

Tests neither run nor reported as not run

Is the issue reproductible on runner?

Test case sample

    @Test(priority = 1)
    public void test1() {
    }
    @Test(dependsOnMethods = "test1")
    public void test2() {
    }
    @Test(dependsOnMethods = "test2")
    public void test3() {
    }
    @Test(dependsOnMethods = "test3")
    public void test4() {
    }
krmahadevan commented 5 years ago

@sdrss the latest released version of TestNG is 7.0.0-beta3. Please retry using that and post back your results.

krmahadevan commented 5 years ago

Attempted at recreating this issue with the latest released version as of today 7.0.0-beta5. Can recreate it.

Test class

import org.testng.annotations.Test;

public class TestClassSample {
  @Test(priority = 1)
  public void test1() {}

  @Test(dependsOnMethods = "test1")
  public void test2() {}

  @Test(dependsOnMethods = "test2")
  public void test3() {}

  @Test(dependsOnMethods = "test3")
  public void test4() {}

}

Listener class

import java.util.ArrayList;
import java.util.List;
import org.testng.ITestListener;
import org.testng.ITestResult;

public class MethodTracker implements ITestListener {

  List<String> log = new ArrayList<>();

  @Override
  public void onTestSuccess(ITestResult result) {
    log.add(result.getMethod().getMethodName());
  }
}

Test runner

import org.assertj.core.api.Assertions;
import org.testng.TestNG;
import org.testng.annotations.Test;

public class IssueTest {

  @Test
  public void reproduceIssue() {
    TestNG testng = new TestNG();
    testng.setTestClasses(new Class[] {TestClassSample.class});
    MethodTracker tracker = new MethodTracker();
    testng.addListener(tracker);
    testng.run();
    Assertions.assertThat(tracker.log).contains("test1", "test2", "test3", "test4");
  }
}

Closing this issue.

sdrss commented 5 years ago

I've managed to test it again on 7.0.0-beta3 and yes it's not reproducible. Thanks @krmahadevan Looking forward for release 7