Configuration from enclosing class not discovered for `@Nested` test class when enclosing class is not annotated with `@ContextConfiguration`

[sbrannen]

Given the following application and test classes, the @Nested test class fails due to getMessage() returning "Hello!" instead of "Mocked!" resulting from the fact that the static nested @TestConfiguration class is only discovered for the top-level enclosing test class.

Specifically, the MergedContextConfiguration for TestConfigurationNestedTests contains classes = [example.Application, example.TestConfigurationNestedTests.TestConfig]; whereas, the MergedContextConfiguration for InnerTests contains only classes = [example.Application].

A cursory search for @TestConfiguration reveals that SpringBootTestContextBootstrapper.containsNonTestComponent(...) uses the INHERITED_ANNOTATIONS search strategy for merged annotations. Instead, it should likely need to make use of TestContextAnnotationUtils in order to provide proper support for @NestedTestConfiguration semantics (perhaps analogous to the use of TestContextAnnotationUtils.searchEnclosingClass(...) in MockitoContextCustomizerFactory.parseDefinitions(...)).

As a side note, if you uncomment @Import(TestConfig.class) both test classes will pass.

package example;

// imports...

@SpringBootApplication
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

    @Service
    public static class GreetingService {
        public String getMessage() {
            return "Hello!";
        }
    }
}

package example;

// imports...

@SpringBootTest
// @Import(TestConfig.class)
class TestConfigurationNestedTests {

    @Test
    void test(@Autowired GreetingService greetingService) {
        assertThat(greetingService.getMessage()).isEqualTo("Mocked!");
    }

    @Nested
    class InnerTests {

        @Test
        void test(@Autowired GreetingService greetingService) {
            assertThat(greetingService.getMessage()).isEqualTo("Mocked!");
        }
    }

    @TestConfiguration
    static class TestConfig {

        @MockBean
        GreetingService greetingService;

        @BeforeTestMethod
        void configureMock(BeforeTestMethodEvent event) {
            when(this.greetingService.getMessage()).thenReturn("Mocked!");
        }
    }

}

[sbrannen]

Actually, if you annotate TestConfig with @Configuration instead of @TestConfiguration, the same behavior is displayed. So it appears that the issue has a larger scope than I originally reported.

[vpavic]

Looks similar to spring-projects/spring-boot#33317, potentially even a duplicate.

[sbrannen]

Thanks for pointing out spring-projects/spring-boot#33317, @vpavic.

It's certainly related, but I wouldn't consider it a duplicate since different solutions will be applied in different places to address the two sets of issues.

[devikaachu]

uncommenting the @Import(TestConfig.class) annotation on the top-level test class will work around the issue.

Additionally, you could consider refactoring your test code to avoid the use of nested @TestConfiguration classes, if possible. For example, you could move the TestConfig class to the top-level test class and use @BeforeEach and @AfterEach methods to set up and tear down the mocked beans.

[wilkinsona]

Thanks, @sbrannen. Boot calls org.springframework.test.context.support.AnnotationConfigContextLoaderUtils.detectDefaultConfigurationClasses(Class<?>), passing in InnerTests and it fails to find TestConfig as a configuration class. I think we'd expected detectDefaultConfigurationClasses to handle this arrangement for us. If that's not the case, is there another API or SPI in the test framework that we should be using instead to avoid reinventing the wheel?

[sbrannen]

Boot calls AnnotationConfigContextLoaderUtils.detectDefaultConfigurationClasses(), passing in InnerTests and it fails to find TestConfig as a configuration class.

That's correct. That utility method only supports finding static nested @Configuration classes for a given test class.

If that's not the case, is there another API or SPI in the test framework that we should be using instead to avoid reinventing the wheel?

As I mentioned in this issue's description, I believe you will need to make use of TestContextAnnotationUtils to support this use case.

The following two test classes demonstrate the difference in behavior between Spring Framework and Spring Boot.

@SpringJUnitConfig
class SpringFrameworkNestedTests {

    @Test
    void test(@Autowired String foo) {
        assertThat(foo).isEqualTo("bar");
    }

    @Nested
    class InnerTests {
        @Test
        void test(@Autowired String foo) {
            assertThat(foo).isEqualTo("bar");
        }
    }

    @Configuration
    static class TestConfig {
        @Bean
        String foo() {
            return "bar";
        }
    }

}

@SpringBootTest
class SpringBootNestedTests {

    @Test
    void test(@Autowired String foo) {
        assertThat(foo).isEqualTo("bar");
    }

    @Nested
    class InnerTests {
        @Test
        void test(@Autowired String foo) {
            assertThat(foo).isEqualTo("bar");
        }
    }

    @Configuration
    static class TestConfig {
        @Bean
        String foo() {
            return "bar";
        }
    }

}

InnerTests in SpringFrameworkNestedTests is successful; whereas, InnerTests in SpringBootNestedTests is not.

Thus, the difference appears to be in the behavior of SpringBootTestContextBootstrapper.

[sbrannen]

Related Issues

• spring-projects/spring-boot#23929

[snicoll]

Thus, the difference appears to be in the behavior of SpringBootTestContextBootstrapper.

I think we agree on that. Andy wrote:

I think we'd expected detectDefaultConfigurationClasses to handle this arrangement for us. If that's not the case, is there another API or SPI in the test framework that we should be using instead to avoid reinventing the wheel?

So. Is there an API we could use? TestContextAnnotationUtils doesn't sound right to me as it is too low-level.

[sbrannen]

So. Is there an API we could use? TestContextAnnotationUtils doesn't sound right to me as it is too low-level.

TestContextAnnotationUtils is the only API we have for honoring @Nested and @NestedTestConfiguration semantics. Please see the Javadoc for details as well existing usage in Spring Framework and Spring Boot for examples.

If you run into any stumbling blocks, let me know, and I'll see if I can help.

[wilkinsona]

Thanks, @sbrannen. I've opened https://github.com/spring-projects/spring-framework/issues/30310 in the hope that things can be improved on the Framework side. In the meantime, I'll see what we can do with TestContextAnnotationUtils but it does feel like reinventing the wheel to me.

[wilkinsona]

This looks like a bug in Spring Framework to me. The difference in behavior described in this comment isn't a difference between Spring Boot and Spring Framework but a difference in Spring Framework's behavior with and without @ContextConfiguration:

package com.example;

import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.test.context.junit.jupiter.SpringJUnitConfig;

import static org.assertj.core.api.Assertions.assertThat;

@SpringJUnitConfig
class SpringJUnitConfigNestedTests {

    @Test
    void test(@Autowired String foo) {
        assertThat(foo).isEqualTo("bar");
    }

    @Nested
    class InnerTests {

        @Test
        void test(@Autowired String foo) {
            assertThat(foo).isEqualTo("bar");
        }

    }

    @Configuration
    static class TestConfig {

        @Bean
        String foo() {
            return "bar";
        }

    }

}

package com.example;

import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.test.context.junit.jupiter.SpringExtension;

import static org.assertj.core.api.Assertions.assertThat;

@ExtendWith(SpringExtension.class)
class SpringExtensionNestedTests {

    @Test
    void test(@Autowired String foo) {
        assertThat(foo).isEqualTo("bar");
    }

    @Nested
    class InnerTests {

        @Test
        void test(@Autowired String foo) {
            assertThat(foo).isEqualTo("bar");
        }

    }

    @Configuration
    static class TestConfig {

        @Bean
        String foo() {
            return "bar";
        }

    }

}

SpringJUnitConfigNestedTests passes because of the @ContextConfiguration meta-annotation and SpringExtensionNestedTests fails because of its absence. With @ContextConfiguration added to the original example or to SpringBootNestedTests they pass as well.

Without @ContextConfiguration, the bootstrapper calls buildDefaultMergedContextConfiguration(testClass, cacheAwareContextLoaderDelegate) which uses Collections.singletonList(new ContextConfigurationAttributes(testClass)) as the config attributes list. With @ContextConfiguration the bootstrapper uses the result of ContextLoaderUtils.resolveContextConfigurationAttributes(testClass) as the config attributes list. In the latter case the @Nested test class becomes the outer class allowing the @Configuration or @TestConfiguration that it encloses to be found.