Listener based Runner

[gege83]

Hi Junit team,

I like your framework it is very useful for me. The only thing what is a bit pain for me is that how to extend your framework with multiple requirement. Please see my extension https://bitbucket.org/gege83/junit-extension/wiki/Home If you can use please notify me.

Thankyou

[sbrannen]

Please note that this is very similar to the TestExecutionListener API supported by the SpringJUnit4ClassRunner and related classes in the spring-test module of the Spring Framework.

I am the author of the Spring TestContext Framework. Thus, if the JUnit team is considering a listener-based API as a plug-in model for JUnit, I would be very happy to participate in such discussions.

[kcooney]

Hi, @sbrannen . Looks like a very clean API. Could you describe some of the benefits of this API outside of spring?

[kcooney]

Looking closer, an API like this is one possible solution to the prolem that people want Rules that act on the class and method level (possibility removing the need for some cases where people create custom runners today)

[sbrannen]

Could you describe some of the benefits of this API outside of spring?

There are in fact quite a few, and I could probably write an entire book on the subject. So I'll do my best to briefly outline some of the major benefits here. ;)

In general, the TestExecutionListener API is very flexible in that it allows developers and third-party frameworks to contribute functionality to tests on an as-needed basis but without limiting the scope of where such a listener can be applied. In this manner, a given listener can perform an action at the class level, method level, and test instantiation level or any combination thereof. Though, typically, one would choose to extend the AbstractTestExecutionListener overriding only those methods required for the task at hand.

For example, Spring's TransactionalTestExecutionListener overrides beforeTestMethod() and afterTestMethod() in order to start and stop a test-managed transaction around the execution of the actual test method (including its @Before and @After methods). As an aside, TransactionalTestExecutionListener provides support for its own @BeforeTransaction and @AfterTransaction annotations, and developers or third-party frameworks often choose the same approach to provide additional functionality via their own annotations.

The TestContextManager is responsible for managing a single TestContext and signaling events to all registered TestExecutionListeners at the execution points defined by the TestExecutionListener API. Furthermore, the TestContextManager ensures that before and after methods of listeners are executed in a wrapped fashion. Thus, a listener that is registered to execute after the TransactionalTestExecutionListener will have its beforeTestMethod() and afterTestMethod() methods executed within the test-managed transaction.

This wrapped ordering allows additional TestExecutionListeners to participate in functionality provided by listeners registered before them in the chain. As a concrete example, both the SqlScriptsTestExecutionListener (provided by Spring) and the DbUnitTestExecutionListener (provided by a third-party) can execute SQL statements that participate with the transaction managed by Spring's TransactionalTestExecutionListener.

All TestExecutionListener implementations provided by Spring are registered automatically, and as of Spring Framework 4.1 third-party frameworks (e.g., Spring Security) can also have their custom TestExecutionListener registered as a default automatically. But... developers can also register listeners explicitly via the @TestExecutionListeners annotation -- for example, to register a custom listener that they've developed for their project. The ordering of listeners can be controlled via the getOrder() method, and custom listeners can be merged with the auto-detected defaults.

Another point of interest is that separate listeners can communicate with each other via the TestContext, since the TestContext implements Spring's AttributeAccessor which essentially allows it to be used as a map for storing key-value pairs.

All of the code in the Spring TestContext Framework (TCF) (aside from the SpringJUnit4ClassRunner and related JUnit-based Statement implementations) can be used with JUnit or TestNG, and hopefully with any other testing framework that becomes mainstream.

In other words, every TestExecutionListener (that I know of) can be used interchangeably with JUnit or TestNG, depending on the preferences of a given development team.

Looking closer, an API like this is one possible solution to the problem that people want Rules that act on the class and method level (possibility removing the need for some cases where people create custom runners today)

That's correct: it could be a viable alternative. However, the TestExecutionListener API is only one piece of the puzzle. Over time we have discovered that the bootstrap mechanism for listeners should also be modifiable by programmatic means in case auto-detection of listeners and declarative configuration via @TestExecutionListeners are insufficient. To this end, we recently introduced a new TestContextBootstrapper SPI in Spring Framework 4.1. A custom bootstrapper, declared via @BootstrapWith, can then override defaults programmatically.

From a design perspective, the only thing Spring-specific about the core internals of the TCF is that the TestContext contains getApplicationContext() and markApplicationContextDirty() methods that are tied to the Spring ApplicationContext.

Most of this is documented in the Testing chapter of the Spring Reference Manual, but if you have further questions, don't hesitate to ask.

Cheers,

Sam

[gege83]

Hi @sbrannen,

You have right. The first idea comes from spring test framework. Unfortunately i was not allowed to use Spring. My only objection is that it tied to Spring. My extension do not use any other dependency injection framework.

[sbrannen]

@gege83,

Aha. That's why it looked so familiar. ;)

Naturally, if JUnit were to adopt such a listener-based API for pluggable extensions, it would have to be generic enough to support any third parties (Spring included). As I mentioned above, the only part of the core of the Spring TestContext Framework (TCF) that is actually tied to Spring (in terms of the TestExecutionListener API and ignoring the implementation details for now) are those two methods in the TestContext API that relate the ApplicationContext.

It's a common misconception that the TCF can only be used with application contexts, dependency injection, and integration tests. This is, however, simply not true. The TCF can in fact be used in unit testing, and more importantly: the TCF works fine without an ApplicationContext (assuming none of the registered TestExecutionListeners attempt to access the application context). In that sense it is a general purpose testing extension framework.

Now, having said all that, I realize that the TCF has a natural tie to Spring since it physically has dependencies on spring-core, spring-context, etc. My point is simply that the TCF is rather generic from a design perspective, and similar APIs and techniques could be applied to JUnit.