Problem using @AfterSuite and dataProvider together

[nguillaumin]

Hi,

I'm trying to get @AfterSuite and dataProvider to work together but I'm always getting a Method testJavaLogFile requires 1 parameters but 0 were supplied in the @Configuration annotation. error.

My use case is that I want a test to be run at the end of our Selenium suite to test that our application logs don't contain any error or exception. I implemented that with:

• A @DataProvider annotated method that provides the name of the log files to check
• A test method that check the content of each log:

@Test(dataProvider="logFiles")
@AfterSuite(alwaysRun=true)
public void testJavaLogFile(String logFile) {
    ...
}

I want this method to run last (hence the @AfterSuite) and I want it to always run regarding of the tests failures (alwaysRun=true). Since I want to use the data provider on it I had to annotate it with @Test as well.

I suspect that's what may be tripping TestNG? The method is correctly executed with the data provider at the end of the run, but TestNG also tries to run it as an individual test due to the @Test annotation and fails because it's missing a parameter?

Is there a way to achieve what I'm trying to do? (@AfterSuite doesn't seem to take a dataProvider attribute). I'm using TestNG v6.8.5 but I'll bump the version to the latest today just to confirm it's still a problem.

Cheers,

Nicolas

[juherr]

@Test(dataProvider="logFiles")
@AfterSuite(alwaysRun=true)
public void testJavaLogFile(String logFile) {
    ...
}

is not legal. You have separate test methods and after* methods.

If you want to have your data provider result, you can store it into the TestContext which is available everywhere.

[nguillaumin]

is not legal

Fair enough, but it's not documented (AFAIK), possible to do and the error message is not very clear, so there may be room for improvement here ;)

I think adding support for @AfterSuite(dataProvider=...) could make sense. An @AfterSuite could be considered a test method after all, so it should support everything a @Test annotated method do?

Thanks for the TextContext pointer, I guess that could work but if I'm not mistaken I'll have to manually iterate over the list of log files, so I'll lose the fact that each log file was treated as a separate test (in the reporting for instance) and the test will fail immediately at the first failing log file (unless I manually take steps to prevent this), that's where the data provider came in handy...

Thanks,

Nicolas

[juherr]

there may be room for improvement here

For sure! But it is the first time I see someone trying to use the same method as a test method + a after method.

An @AfterSuite could be considered a test method after all, so it should support everything a @Test annotated method do?

I'm not an expert of the theory, but before and after methods are not tests. Before methods are often used to prepare the tests, and after methods are often used to stop what before methods have just started. @cbeust What is your opinion?

I manually take steps to prevent this

Yes, why not?

[nguillaumin]

Thanks for your reply.

Yes, why not?

I definitely can, but the data provider would be nice to use here to have separate reporting on each test instance, rather than having to manually build a list of "failures" and assert a single time at the end.

[juherr]

Your test design (an after method as a test) is weird for me, but I propose you describe it on the mailing list and/or stackoverflow to have another point of view than mine :)