search

randomizedtesting / randomizedtesting

Randomized Testing (Core JUnit Runner, ANT, Maven)
http://labs.carrotsearch.com/randomizedtesting.html
Apache License 2.0
173 stars 42 forks source link

Listeners can only subscribe to aggregated events #141

Closed cchabanois closed 11 years ago

cchabanois commented 11 years ago

I wanted to create a custom listener, not for reporting but to send test progress to another process. But I can't subscribe to events like TestStartedEvent ...

dweiss commented 11 years ago

Yeah, these are internal and are not propagated to listeners. The problem is that these events lack the forked VM identifier so you wouldn't know which VM a test is running on, for example (and in case of multiple duplicated test suites this could be a problem).

Can you share some light what is your overall scenario? Maybe there is a workaround that doesn't involve fine-grained events that I could suggest?

cchabanois commented 11 years ago

To know progress I need to know the test hierarchy (suites and tests) and then for each test : when it starts, when it ends or fails. Each test must have an identifier

dweiss commented 11 years ago

You didn't explain what you are up to, you just said what you need. JUnit tests don't have unique identifiers. Check out the Description class -- this is the only thing available about a test. It's got nothing to do with this runner, actually, it's just by-design.

cchabanois commented 11 years ago

I want to do what is done in eclipse when you run a junit test suite : display the progress as tests are run. I managed to do that with a junit4 RunListener. I associate each Description with an unique id. I know this is a problem if test is present more than once in the test suite but it's ok for now (there is the same problem in eclipse anyway : try a suite using the same test several times). But the standard junit ant task cannot take a RunListener as parameter (and ant's JUnitResultFormatter is limited). That's why I tried junit4 ant task but it cannot use a RunListener and aggregated events will not do the job.

dweiss commented 11 years ago

You should have started from that. We can't really count individual tests because the task doesn't know how many there will be (this information is only available on the forked JVM, after a suite has been initialized which is done lazily and has to stay this way) but we can determine how many suites have already been executed and how many are still left. So you could have a crude progress over entire suites but not over individual tests.