search

testng-team / testng

TestNG testing framework
https://testng.org
Apache License 2.0
1.99k stars 1.02k forks source link

IConfigurationListener is not executed and IDataProviderListener is not added to the list of the listeners if a new listener is added at the runtime #2043

Closed baflQA closed 5 years ago

baflQA commented 5 years ago

TestNG Version

7.0.0-beta3

Expected behavior

Listener implementing IInvokedMethodListener, IDataProviderListener and IConfigurationListener added onSuiteStart is executed during the tests execution

Actual behavior

Only IInvokedMethodListener is executed.

Is the issue reproductible on runner?

Test case sample

Hi folks! There is a given listener: https://github.com/baflQA/testNG_parallel_debug/blob/master/src/main/java/testngparallel/listeners/FailFastListener.java

In the class implementing ISuiteListener, I'm adding this listener in "onStart" method: https://github.com/baflQA/testNG_parallel_debug/blob/master/src/test/java/testngparallel/MasterTestClass.java#L21

When running the test: https://github.com/baflQA/testNG_parallel_debug/blob/master/src/test/java/testngparallel/Class1.java

I'm setting the breakpoints in the FailFastListener methods. Only the "afterInvocation" method of IInvokedMethodListener is executed. In https://github.com/cbeust/testng/blob/master/src/main/java/org/testng/SuiteRunner.java#L449 I see that IDataProviderListener is not registered at all.

But IConfigurationListener is registered https://github.com/cbeust/testng/blob/master/src/main/java/org/testng/SuiteRunner.java#L463 so I would expect "beforeConfiguration" method to be executed as well.

Can You pls help me understand if there are/is bug(s), or it's by design? My goal is to register this listener at the runtime. Thx for Your help!

baflQA commented 5 years ago

@juherr @krmahadevan thx for Your effort! But it's not fixed as long as it's not released ;) Any plans for another beta or the final version rollout? There are plenty of nice fixes since last update, would be nice to public them.

juherr commented 5 years ago

Yes, we should release a new beta.

baflQA commented 5 years ago

@krmahadevan I've spotted that the ITestListener is missing there as well. Before I open a new issue, can You pls tell me if it's by design or not? I would like to conditionally register some listener executed before/after the test.

krmahadevan commented 5 years ago

Hard to say without looking at the code. Can you please open up a new issue which captures all the cases ? That way we can address all of them in one shot. As always, please dont forget to include a sample that can be used to reproduce the problem.