search

randomizedtesting / randomizedtesting

Randomized Testing (Core JUnit Runner, ANT, Maven)
http://labs.carrotsearch.com/randomizedtesting.html
Apache License 2.0
173 stars 42 forks source link

Events for @BeforeClass and @AfterClass in JSON output #61

Open stanislawosinski opened 12 years ago

stanislawosinski commented 12 years ago

If the execution of @BeforeClass and @AfterClass code was marked in the executionEvents array, we'd be able to insert the right markers in the console output report. Currently, there's no reliable way to correctly attribute the non-test output events.

dweiss commented 12 years ago

Sadly, there is no way to do it in the general case without altering JUnit. These are simply transparent -- there is even no way to easily distinguish class level and instance level hooks (or rules). We could add it in our own runner but it wouldn't be generally available for other runners.

dweiss commented 12 years ago

I'm moving this to 2.0.0, but I think this isn't going to be possible without altering JUnit internals (or using runtime code transformations like aspect weaving).

stanislawosinski commented 12 years ago

Ok, let's leave it for now. I could extrapolate this information to some extent, but it could be misleading.

dweiss commented 12 years ago

No, don't try to estimate anything -- this will be confusing. The timestamps of "here the test started and here it ended" are all right and correct. The console output is complete (or should be!); if there's out-of-context output then it's a pity; somebody will have to correct his or her tests, we shouldn't try to cater for entirely broken code.