Not all methods of Springs `TestContextManager` are invoked.

[mpkorstanje]

👓 What did you see?

Not all methods of Springs TestContextManager are invoked.

✅ What did you expect to see?

To facilitate integration testing frameworks such as Cucumber, Spring provides a TestContextManager to help setup a Spring application for tests. However currently not all methods are invoked, or invoked with partial information.

TestContextManager is the main entry point into the Spring TestContext Framework. Specifically, a TestContextManager is responsible for managing a single TestContext and signaling events to each registered TestExecutionListener at the following test execution points.

• before test class execution: prior to any before class callbacks of a particular testing framework (e.g., JUnit 4's @BeforeClass)
• test instance preparation: immediately following instantiation of the test class
• before test setup: prior to any before method callbacks of a particular testing framework (e.g., JUnit 4's @Before)
• before test execution: immediately before execution of the test method but after test setup
• after test execution: immediately after execution of the test method but before test tear down
• after test tear down: after any after method callbacks of a particular testing framework (e.g., JUnit 4's @After)
• after test class execution: after any after class callbacks of a particular testing framework (e.g., JUnit 4's @AfterClass)

Cucumber currently does not invoke test instance preparation, before test execution, after test execution at all. Cucumber can not invoke these methods with the correct semantics because the ObjectFactory is not aware of the test execution and can not execute anything between the execution of before scenario hooks and step definitions. Further more, the ObjectFactory is not aware of aware of the result of any scenario and can not pass any exceptions thrown to the after- methods.

:classical_building: What would be a possible solution?

After #2174 it should be possible to refactor Cucumbers internals such that instead of providing providing Backend and ObjectFactory extension points, we can provide individual extension points equivalent to JUnit Jupiters: BeforeAllCallback, AfterAllCallback, TestInstancePostProcessor, BeforeEachCallback, AfterEachCallback, BeforeTestExecutionCallback and AfterTestExecutionCallback. To ensure these callbacks can transfer state an equivalent to JUnit Jupiters ExtensionContext would also have to be provided. Specifically the ObjectFactory could be replaced with the TestInstanceFactory and Backend could be reduced to only the discovery of glue definitions.

Additional benefits of closely following this pattern would be the relative ease by which we can re-implement existing extensions from JUnit JUpiter for Cucumber. We can benefit from a larger ecosystem here.

Do note: A 1-to-1 adoption of JUnit Jupiters extension system is not possible. JUnit Jupiter assumes that there is a single test class, test instance, and method. Cucumber generally has multiple.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in two months if no further activity occurs.