IExecutionListener.onStart() running twice when used as annotation in mediumSuite file

[deeptiGuptaUIUC]

TestNG Version: 6.14.3

Note: only the latest version is supported

Expected behavior:

The documentation states that in a testng run, IexecutionListener will be executed only once.

Actual behavior:

When we use IexecutionListener as annotation in test cases and bundle those suites to run using a XML, the onStart() function gets executed multiple times. In a medium suite, it is executed 2 times. In a large suite file, which contains 2 other suites (small and medium), the onStart function is executed 3 times.

Is the issue reproductible on runner?

• [ ] Shell
• [ ] Maven
• [ ] Gradle
• [ ] Ant
• [X] Eclipse
• [ ] IntelliJ
• [ ] NetBeans

Test case sample

Please, share the test case (as small as possible) which shows the issue

Sample Test Class

import static org.testng.Assert.assertEquals;

import org.testng.annotations.Listeners;
import org.testng.annotations.Test;
@Listeners(listeners.ExecutionListener.class)
public class TestClass5 {

    @Test
    public void test1() {
        assertEquals(true, true);
    }

    @Test
    public void test2() {
        assertEquals(true, true);
    }

    @Test
    public void test3() {
        assertEquals(true, true);
    }

    @Test
    public void test4() {
        assertEquals(true, true);
    }

}

Sample XML File

<?xml version = "1.0" encoding = "UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >

<suite name="Suite">
    <test name="exampletest3">
        <classes>
            <class name="suite3.TestClass5" />
            <class name="suite3.TestClass6" />
        </classes>
    </test>
    <suite-files><suite-file path="src/test/resources/smallSuite.xml" /></suite-files>
    <suite-files><suite-file path="src/test/resources/mediumSuite.xml" /></suite-files>
</suite>    

Sample Output

Start
Start
Start

===============================================
smallSuite
Total tests run: 8, Failures: 0, Skips: 0
===============================================

===============================================
mediumSuite
Total tests run: 8, Failures: 0, Skips: 0
===============================================

===============================================
Suite
Total tests run: 24, Failures: 0, Skips: 0
===============================================

Finish

Sample ExecutionListener:

import org.testng.IExecutionListener;

public class ExecutionListener implements IExecutionListener{

    public void onExecutionStart() {
        System.out.println("Start");
    }

    public void onExecutionFinish() {
        System.out.println("Finish");
    }
}

[formynet]

in which release of testng is this bug fix available?

[formynet]

@cbeust do you know in which release of testng will this bug fix be available?

[krmahadevan]

@formynet - Please make use of the latest released version of TestNG viz., 7.0.0-beta5