Configuration methods are invoked multiple times when using groups programatically

[dagansandler]

When setting an included group on test or suite level, configuration methods are running multiple times instead of just once.

TestNG Version

7.1.0

Expected behavior

Output:

before suite
before test
before class
before method
test testMethod
after method
after class
after test
after suite

and before/after invocation methods in listener should only be called 9 times each

Actual behavior

Output:

before suite
before test
before class
after class
after method
after suite
after test
before class
before method
before suite
before test
before method
test testMethod
after method
after test
before test
after class
after suite
before suite
after method
before class
before method
after class
after test
after suite

and before/after invocation methods in listener are called 25(!) times

Is the issue reproductible on runner?

The issue is reproducible when using the TestNG APIs to construct a xml suite programatically.

Test case sample

Test class:

package test;

import org.testng.annotations.*;

@Test(groups = "A")
public class MyTest {

    @BeforeSuite
    public void beforeSuite() { System.out.println("before suite"); }

    @BeforeTest
    public void beforeTest() { System.out.println("before test"); }

    @BeforeClass
    public void beforeClass() { System.out.println("before class"); }

    @BeforeMethod
    public void beforeMethod() { System.out.println("before method"); }

    @Test
    public void testMethod() { System.out.println("test testMethod"); }

    @AfterMethod
    public void afterMethod() { System.out.println("after method"); }

    @AfterClass
    public void afterClass() { System.out.println("after class"); }

    @AfterTest
    public void afterTest() { System.out.println("after test"); }

    @AfterSuite
    public void afterSuite() { System.out.println("after suite"); }

}

Main class:

package test;

import org.testng.*;
import org.testng.xml.*;

import java.util.ArrayList;
import java.util.Collections;
import java.util.List;

public class Main {

    public static void main(String[] args) {

        List<XmlPackage> packages = new ArrayList<>();
        XmlPackage xmlPackage = new XmlPackage("test");
        packages.add(xmlPackage);

        XmlTest test = new XmlTest();
        test.setName("MyTest");

        XmlSuite xmlSuite = new XmlSuite();
        xmlSuite.setName("MySuite");
        xmlSuite.setTests(Collections.singletonList(test));
        xmlSuite.setXmlPackages(packages);

        xmlSuite.addIncludedGroup("A");  // Everything works properly if this line is removed
        // test.addIncludedGroup("A"); // Alternatively, setting group on test level also causes failure.

        test.setXmlSuite(xmlSuite);

        MyInvokedMethodListener listener = new MyInvokedMethodListener();
        TestNG tng = new TestNG();
        tng.addListener(listener);
        tng.setXmlSuites(Collections.singletonList(xmlSuite));
        tng.setVerbose(1);
        tng.run();

        Assert.assertEquals(listener.beforeCount, 9);
        Assert.assertEquals(listener.afterCount, 9);
    }

    public static class MyInvokedMethodListener implements IInvokedMethodListener {
        int beforeCount = 0;
        int afterCount = 0;

        @Override
        public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
            beforeCount++;
        }

        @Override
        public void afterInvocation(IInvokedMethod method, ITestResult testResult) {
            afterCount++;
        }
    }

}

[tracey-ruark]

We're also able to reproduce this issue using version 7.1.0 across all our teams and various testing projects. Nobody has reported any such issues running with 6.14.3.

As dagansandler describes in his example code this can be reproduced by simply copying his example test class into a maven project then run $ mvn test -Dgroups=A.

[krmahadevan]

This is a duplicate of https://github.com/cbeust/testng/issues/2209

Please follow/comment on the original issue to track this.