search

spring-projects / spring-framework

Spring Framework
https://spring.io/projects/spring-framework
Apache License 2.0
56.47k stars 38.09k forks source link

Introduce annotation to disable test based on active Spring profile [SPR-11677] #16300

Open spring-projects-issues opened 10 years ago

spring-projects-issues commented 10 years ago

Torsten Krah opened SPR-11677 and commented

I've looked at the @IfProfileValue and @ActiveProfiles annotations, but neither does what I am searching for.

We are using the "spring.profiles.active" system property to activate different profiles, even at the test level -- e.g. "h2, default" or something else.

We have tests which are only for Oracle and should therefore only be run if the "oracle" profile is active, but I've found no way to express this on the test class itself.

A dedicated annotation would be nice to support this.

Affects: 4.0.3

Issue Links:

8 votes, 8 watchers

spring-projects-issues commented 10 years ago

Sam Brannen commented

Hi Torsten,

I've linked several related issues: #9538, #13622, #13625, #12410.

Can you please review them to see if your needs would be addressed by one of them?

Thanks,

Sam

spring-projects-issues commented 10 years ago

Torsten Krah commented

If i read all of the issues right, they are about setting the profile in a test, i did not found a hint how to actually trigger the test class at all based on the active profile (which i did set already via System property when running the test suite @jenkins) - like @Ignore on the class or method - something like @RunWithSpringProfile("h2", "default") - or something.

9538 seems to be the same direction but has no solution yet. Feel free to correct me if i missed some comment at one task you've linked.

Thanks, Torsten.

spring-projects-issues commented 10 years ago

Christoph Strobl commented

Hi,

We've had similar requirements when testing against different versions. Basically we created a custom JUnit @Rule to disable tests for features only available in Redis 2.8 when running against 2.6.

Let me outline what I am thinking of in terms of dealing with spring profiles.

I've added a sample below basically adding a new annotation IfSpringProfileActive holding the desired profile name. The SpringProfileRule checks for the presence of IfSpringProfileActive, loads the current Environment if not already set, and verifies that the desired profile is currently active.

There's SpringRuleTest at the end of the sample to show what it does.

/**
 * Annotation holding name of desired profile used to mark methods from being potentially excluded. 
 */
@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface IfSpringProfileActive {

    String value();
}
/**
 * TestRule verifying Tests marked with IfSpringProfileActive are verified against currently active spring profile
 */
public class SpringProfileRule implements TestRule {

    private Environment env;

    private SpringProfileRule(Environment env) {
        this.env = env;
    }

    public static SpringProfileRule forSpringJunitClassRunner() {
        return new SpringProfileRule(null);
    }

    public static SpringProfileRule forEnvironment(Environment env) {
        return new SpringProfileRule(env);
    }

    @Override
    public Statement apply(final Statement base, Description description) {

        IfSpringProfileActive profileValue = description.getAnnotation(IfSpringProfileActive.class);
        final String requiredProfile = profileValue != null ? profileValue.value() : null;

        return new Statement() {

            @Override
            public void evaluate() throws Throwable {

                if (StringUtils.hasText(requiredProfile)) {
                    initEnvironmentWhenNotSet(base);
                    verify(requiredProfile);
                }

                base.evaluate();
            }
        };

    }

    protected void initEnvironmentWhenNotSet(Statement base) {

        if (env == null && base instanceof RunAfterTestMethodCallbacks) {

            // there should be a better way of doing this...
            TestContextManager contextManager = (TestContextManager) new DirectFieldAccessor(base)
                    .getPropertyValue("testContextManager");
            TestContext testContext = (TestContext) new DirectFieldAccessor(contextManager).getPropertyValue("testContext");
            env = testContext.getApplicationContext().getEnvironment();
        }
    }

    protected void verify(String requiredProfile) throws Throwable {

        if (StringUtils.hasText(requiredProfile)) {
            if (!env.acceptsProfiles(requiredProfile)) {
                throw new AssumptionViolatedException(String.format("Profile %s is currently not active", requiredProfile));
            }
        }
    }

}
/**
 * A quick test to show that it works
 */
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration
@ActiveProfiles(profiles = "oracle")
public class SpringRuleTest {

    @Autowired String foo;

    public @Rule SpringProfileRule rule = SpringProfileRule.forSpringJunitClassRunner();

    @Configuration
    static class Config {

        @Bean
        String foo() {
            return "foo";
        }
    }

    @Test
    public void shouldAlwaysRun() {
        System.out.println("Run no matter what!");
    }

    @Test
    @IfSpringProfileActive("oracle")
    public void shouldOnlyRunWhenActiveProfileSetToOracle() {
        System.out.println("Runnig on oracle.");
    }

    @Test
    @IfSpringProfileActive("not-oracle")
    public void shouldBeIgnoredIfActiveProfileSetToOracle() {
        System.out.println("Oracle should not see me.");
    }

}
spring-projects-issues commented 9 years ago

Caleb Cushing commented

Why not just allow test methods and test classes to be annotated with @Profile and skip them based on the same rules beans would otherwise not be processed with?

This is the most obvious solution to me, and is the first solution I reached for.

tauinger-de commented 3 years ago

As mentioned above annotating test classes and methods with @Profile is a possible solution and IMHO the most intuitive way. Basically I did that without thinking and was wondering why it doesn't work.

sbrannen commented 9 months ago

Please note that it is already possible to enable/disable a test class or test method via a SpEL expression when using JUnit Jupiter (from JUnit 5).

Inspired by the test case supplied by @christophstrobl, I put together the following example.

import org.junit.jupiter.api.Test;
import org.springframework.context.annotation.Configuration;
import org.springframework.test.context.ActiveProfiles;
import org.springframework.test.context.junit.jupiter.EnabledIf;
import org.springframework.test.context.junit.jupiter.SpringJUnitConfig;

@SpringJUnitConfig
@ActiveProfiles("oracle")
class SpringProfileTests {

    @Test
    @EnabledIf(expression = "#{environment.matchesProfiles('oracle')}", loadContext = true)
    void shouldOnlyRunWhenActiveProfileSetToOracle() {
        System.out.println("Runnig on oracle.");
    }

    @Test
    @EnabledIf(expression = "#{environment.matchesProfiles('!oracle')}", loadContext = true)
    // @DisabledIf(expression = "#{environment.matchesProfiles('oracle')}", loadContext = true)
    void shouldBeIgnoredIfActiveProfileSetToOracle() {
        throw new RuntimeException("Oracle should not see me.");
    }

    @Configuration
    static class Config { /* no beans */ }
}

The test class passes as-is, but if you set the active spring profile to not-oracle then shouldOnlyRunWhenActiveProfileSetToOracle() will be disabled, and shouldBeIgnoredIfActiveProfileSetToOracle() will be enabled and fail.

The key thing to note is the use of the @EnabledIf/@DisabledIf annotations from spring-test (not the @EnabledIf/@DisabledIf annotations from junit-jupiter-api).

@EnabledIf(expression = "#{environment.matchesProfiles('oracle')}", loadContext = true)

Another very important thing to note is that it only works with loadContext = true, and that means that the test's ApplicationContext will be eagerly loaded even if it's potentially never needed.

The latter is a downside to any approach I can think of to reliably "disable a test based on an active Spring profile".

Using @Profile on test classes or test methods -- as suggested in various comments in this issue -- would have the same downside.

sbrannen commented 9 months ago

The latter is a downside to any approach I can think of to reliably "disable a test based on an active Spring profile".

The benefit of using something like the @EnabledIf example above with a SpEL expression is that it matches bean definition profiles that have been registered in the Environment for the test's actual ApplicationContext, and that's what I meant by "reliable".

However, if all you want to do is match against the value of the spring.profiles.active system property, you can do that with JUnit Jupiter's @EnabledIfSystemProperty / @DisabledIfSystemProperty support.

The following example demonstrates how to achieve that.

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.condition.DisabledIfSystemProperty;
import org.junit.jupiter.api.condition.EnabledIfSystemProperty;
import org.springframework.context.annotation.Configuration;
import org.springframework.test.context.junit.jupiter.SpringJUnitConfig;

// Run test class with -Dspring.profiles.active=oracle
@SpringJUnitConfig
class SystemPropertyProfileTests {

    @Test
    @EnabledIfSystemProperty(named = "spring.profiles.active", matches = "oracle")
    void shouldOnlyRunWhenActiveProfileSetToOracle() {
        System.out.println("Runnig on oracle.");
    }

    @Test
    @DisabledIfSystemProperty(named = "spring.profiles.active", matches = "oracle")
    void shouldBeIgnoredIfActiveProfileSetToOracle() {
        throw new RuntimeException("Oracle should not see me.");
    }

    @Configuration
    static class Config { /* no beans */ }
}
sbrannen commented 9 months ago

Note that Environment.matchesProfiles(String...) supports profile expressions.

Thus, the following:

@EnabledIf(expression = "#{environment.matchesProfiles('!oracle')}", loadContext = true)

is equivalent to:

@DisabledIf(expression = "#{environment.matchesProfiles('oracle')}", loadContext = true)

I've updated the example in https://github.com/spring-projects/spring-framework/issues/16300#issuecomment-1884700274 to highlight that.