TNG / junit-dataprovider

A TestNG like dataprovider runner for JUnit with many additional features
Apache License 2.0
246 stars 164 forks source link

FrameworkMethod didn't get the right test #134

Closed laaron2008 closed 2 years ago

laaron2008 commented 2 years ago

my test class likes this image

my dataprovider likes this image

when I run the getProduct case, the method is not getProduct, why?

aaschmid commented 2 years ago

First guess: This is most likely due to the test names which are adjusted for working with data providers. What is the output of method.getName()?

laaron2008 commented 2 years ago

it's getDuplicationProductDetail

aaschmid commented 2 years ago

Ok, understood. This is a problem because before we know which test cases are available, we need to execute the dataProvider. The result of the ´dataProvider` is however cached and therefore only called once in order to do not waste time. In your case with the wrong name :-(

Can you try switching the cache off using @DataProvider(cache = false) instead?

laaron2008 commented 2 years ago

I tried, but got the same result. I just added the @DataProvider(cache = false) Is there anything else I need to do?

aaschmid commented 2 years ago

No, this should do the trick. The dataProvider method should now be called multiple times, once for each test method referencing it.

Then, I need some more time and try to get a reproducer for the issue.

aaschmid commented 2 years ago

I am sorry @laaron2008 but I cannot reproduce it using cache = false. Here is my reproducer https://github.com/aaschmid/junit-dataprovider--134. Executed the tests from within IntelliJj or ./gradlew -i --no-build-cache clean test --tests "MyTest.getProduct[*0: {}, {}*]" on command line.

laaron2008 commented 2 years ago

I changed https://github.com/aaschmid/junit-dataprovider--134, and reproduce the problem. image image when I run the getProduct case, the method.getName() printed getDuplicationProductDetail

laaron2008 commented 2 years ago

I guess the problem is

MyTest extends BaseJunit

aaschmid commented 2 years ago

Even though I would not suggest adding the @RunWith(DataProviderRunner.class) on BaseJunit I cannot reproduce it (see updates on https://github.com/aaschmid/junit-dataprovider--134). What did I do forget?

Bildschirmfoto 2021-12-12 um 09 15 52 Bildschirmfoto 2021-12-12 um 09 15 57

Reason why I would not inherit from BaseJunit: because e.g. the IDE thinks you can execute it as tests which fails with org.junit.runners.model.InvalidTestClassError: Invalid test class 'BaseJunit' if there are not real test cases in there, e.g. if you run all test cases. And if there are testcases in, they will be executed with every child test class once...

aaschmid commented 2 years ago

Same in IntelliJ

Bildschirmfoto 2021-12-12 um 09 22 59
laaron2008 commented 2 years ago

I gave up inherit and the case worked, thanks

aaschmid commented 2 years ago

Ack, so I will close the issue. Don't hesitate to re-open it.