Investigate methods of restarting applications for faster TDD

This is a feature-request.

Any non-trivial (web + any datastore) spring boot application would have a 10s seconds startup time, most of them around 1 minute, and some extreme cases reach to 2 minutes. This is obviously a huge blocker for any flavor of test driven development work-style. Anything beyond a few seconds to run the test and get a feedback on my changes makes the TDD style impossible to employ. I'm sure many smart people already thought and pointed out this but the fact that no official solution or recommended pattern is known (to the best of my knowledge) is surprising to me.

Here's the use case: I would like to employ a TDD approach; one line of test code, run the test, one line of production code, run the test, repeat with minimal waiting time for Spring context to start.

Solutions

One solution is to keep the context light. I'm pretty sure there's a limit to this and very quickly it hits the ceiling in terms of startup performance optimization. A better solution is to start the application once (in the IDE or in command line using Gradle for example) and start coding. Having the DevTools, production codes are reloaded automatically very quickly and test code can be executed as many times as needed considering that now the we don't have to wait for the context to start. I've created a proof of concept here.

Proposal (Draft)

SpringBootTestApplication Annotation How to use it: Add it to a (non-test) class in the test folder which effectively makes the class a spring boot application and use the class as the configuration class in SpringBootTest annotation wherever the context is needed. What it does: when added to a class, creates a very thin Spring Boot application in which it creates the main application (or a slightly different version of it with respect to main application configurations) as a bean.

Vision for the feature

Create a thin spring boot application (container for the under-test main application)
Create the under-test main application with the given custom configurations and properties (to make the main application testable, like mocks, etc)
Propagate environments, profiles, and properties from the test to the under-test application. (whatever that means!)
Provide an easy way to access beans defined in the under-test main application (e.g. JDBC DataSource)
Handle shared resource properly between different instances of under-test main application (e.g. ports, test-containers, etc) if the user requires more than one version of the main application in different tests.

Notes

Maybe the same functionality can be achieved by child context, but I'm not sure how much it'd be IDE and Gradle/Maven friendly. This definitely is an objective for this feature in my eyes.

I'd be happy to work on the feature, but for sure needs help with some of the internal code of Spring Boot Test module. Thanks.

Thanks for the proposal.

When practicing TTD on a Spring app, I find that I make minimal use of full-blown integration tests when in the red, green, refactor loop. Much of the time, I'll be working on an individual unit running unit tests that have no dependency on Spring at all. These tests are very quick. For the less common occasions when I'm working on an area of integration between multiple units, I find sliced tests useful as they allow me to focus the integration test on only a small number units. These tests can also be quick although not as quick as the unit tests. The slowest part is often starting Docker containers via Testcontainers, but this can be improved with container reuse. The longer-running @SpringBootTest-based tests can then be saved for broader integration testing of the whole app. I typically run those once outside of the TDD loop prior to pushing.

I was hoping we don't get into this discussion 😁 OK let me try. Considering everything equal, for any non-trivial feature:

Writing integration tests generally is easier. Especially for features with too many dependencies -> too many mocks. For the same reason, maintaining unit tests is generally harder since they depend on too many mocks (which makes them brittle).
Integration tests are more reliable for obvious reasons.

Now one could argue against the above by saying none of the above is true for a bad integration test. To which my reply would be if someone writes a bad integration test, they would also write bad unit tests too. So "everything equal", still the above hold true. Or at the very least you cannot make any conclusion about which type of test is better. I also skip "the risk of testing mocks instead of production code" in unit tests to not open another blackhole-like discussion 🙂 (I also fully appreciate the significance of unit test in places where the scope of the test coverage is low. No doubt!) If we agree about the above, or if at least they make sense, I think it's a relevant question, "I write more unit tests than integration test" is it not (mainly) because integration tests are harder to implement and maintain (because they are slow) or is there really huge value in writing unit test? I don't think you would disagree with "there's nothing inherently superior with unit tests". If anything, theoretically we could have been in a much better place, had we been able to cover everything with integration tests. Unfortunately that's not possible. In fact this is the recommendation in Micronaut community (to the best of my knowledge); write more integration tests than unit tests, because Micronaut application starts fast, so there's no need to replace them with unit tests. This is so huge in my eyes that, (for me personally) it could be a valid reason to choose Micronaut over Spring Boot, despite all the cost associated with this choice (limited components, features, documentation, experience, etc). (Especially now that Spring is adding native image support 🙂) Now back to the main point. Even if we could not agree on all of the above, I believe there's a good number of developers appreciate faster integration tests in their Spring Boot applications. (The fast that this feature got never discussed/implemented could be a sign that I'm wrong, but who knows 🙂). Probably personal style of development also plays a role here 🤷 Hopefully this makes sense. But happy to discuss further if it does not. Thanks.

Of course, writing code is not easy. But When it's done using Test driven development methods it will look easier and totally hassle-free. Actually, the solution here is Integration tests. Also, FYI I've also made an article on Test driven development; please visit it if any of our curious readers want to know more interesting facts about Test Driven Development.

Is https://github.com/spring-projects-experimental/spring-boot-testjars a relevant work? Currently what I'm doing is to spin up the main boot app using a separate Gradle task and run tests (which are a minimal spring boot application) against the main app. This way I can write tests using Spring Boot features but since is a bare minimum app, they'll run pretty fast and since they depends on the main application, if there's a need for the app to restart (due to any production code change) they'll wait until the main app is ready and then they proceed. Works pretty good so far. The only major limitation· is that I don't have access to major beans in the main app; e.g. repositories. Than means I need to duplicate some piece on the test boot app side.

spring-projects / spring-boot