testng-team / testng

TestNG testing framework
https://testng.org
Apache License 2.0
1.98k stars 1.02k forks source link

Listener that implements IMethodInterceptor is used when it should not be #1195

Open kodayashi opened 7 years ago

kodayashi commented 7 years ago

TestNG Version

6.9.13.6

Expected behavior

When annotating a class (that contains three tests) with a listener that implements IMethodInterceptor, and using a testng xml file that specifies that class, the "intercept" method is run.

When using a testng xml file that specifies a different class that is not annotated with the same listener, the "intercept" method for that listener should not be executed as the listener shouldn't be bound to the class.

Actual behavior

The intercept method is indeed executed. Somehow this listener is being activated even though it only annotates a class that is out of scope from test execution.

Interestingly, if the java annotation is removed, and the listener is specified only in the xml file for the class that it should be enabled for, everything works as expected.

Is the issue reproductible on runner?

In the example below, the CustomListener class' intercept method is run for testA(), even though the annotation is only in the TestB class:

Custom Listener

public class CustomListener implements IMethodInterceptor {

  @Override
  public List<IMethodInstance> intercept(List<IMethodInstance> methods, ITestContext context) {  System.out.println("Hello there!"); return new ArrayList<IMethodInstance>(); }

Test Class A

public class TestA extends MyBaseTestClass {

    @Test(groups = { TestConstants.TestNGGroups.GROUP_A }, description = "Group A.")
    public void testA() { /* Do stuff and Assert */ }
}

Test Class B

@Listeners(CustomListener.class)
@Test(groups = { TestConstants.TestNGGroups.GROUP_B }, description = "Group B.")
public class TestB extends MyBaseTestClass {

    public void testB() { /* Do stuff and Assert */ }
}

Testng File

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="Test Suite A" verbose="1" parallel="classes"
    thread-count="1">
  <test name="Test A" preserve-order="true">
        <groups>
            <run>
                <include name="group_a" />
            </run>
        </groups>
        <packages>
        <package name="com.mycompany.tests" />
        </packages>
  </test>
</suite>
juherr commented 7 years ago

Listeners are global for all the suite and not scoped for a specific class, even with the annotation.

But I'm agree, the listeners on an excluded class should be skipped.