testng-team / testng

TestNG testing framework
https://testng.org
Apache License 2.0
1.98k stars 1.02k forks source link

Duplicate Tests being added to XmlTests if XmlTest is used in the parameters of annotated methods #1994

Open ApexK opened 5 years ago

ApexK commented 5 years ago

TestNG Version

v7

Expected behavior

Current XmlTest is returned

Actual behavior

Duplicate test is added to XmlTest

Is the issue reproductible on runner?

Test case sample

For example, use the following @BeforeTest

@BeforeTest(alwaysRun = true)
@Parameters({ "browser"})
 public void beforeTest(@Optional XmlTest xmlTest, @Optional String browser) {
      System.setProperty("webdriver.chrome.driver","/path/to/chromedriver");
      DesiredCapabilities c=DesiredCapabilities.chrome();
      ChromeOptions options = new ChromeOptions();
      options.addArguments("options");
}

Run any sample suite with around 5 test cases Now after the tests are executed, check the total number of tests using debugging mode in m_tests. It shows 8 tests which include duplicate tests being added which contain empty xmlClasses image

These duplicate/ empty test cases are also causing random NullPointer Exceptions at the end of the tests. These NPEs are however not reproducible all the time.

Attached is a small zip file with a sample maven project. You can reproduce the duplicate tests issue by just running the testng.xml file from eclipse and checking the m_tests at the AfterSuite methods by using ITestContext like the below:

@AfterSuite
public void afterSuite(ITestContext context) {
      System.out.println("afterSuite");
}

Please let me know if any other info is needed

DuplicateTests.zip

ApexK commented 5 years ago

@krmahadevan I don't mean to rush but would be nice to know if there has been any update on this?

krmahadevan commented 5 years ago

@ApexK - I have raised a PR to fix this issue. But me and @juherr haven't been able to arrive at a consensus on what we think would be the right way of fixing this.

@cbeust - Can you please chime in and provide your insights. I would really like to close this PR Need your comments here