Reduce the boilerplate that's required in tests that use a service in a Testcontainers-managed container

[wilkinsona]

A common pattern when using Testcontainers is to use @DynamicPropertySource to configure the properties required to use the service in the Testcontainers-managed container. For example, you might do something like this to use Redis:

@Container
static GenericContainer redis = new GenericContainer(DockerImageName.parse("redis").withTag("4.0.14"));

// …

@DynamicPropertySource
static void redisProperties(DynamicPropertyRegistry registry) {
    registry.add("spring.data.redis.host", redis::getHost);
    registry.add("spring.data.redis.port", redis::getFirstMappedPort);
}

We can reduce this boilerplate by automatically extracting connection details from the service in the container and making them available to the auto-configuration:

@Container
@RedisServiceConnection
static GenericContainer redis = new GenericContainer(DockerImageName.parse("redis").withTag("4.0.14"));

Here, @RedisServiceConnection indicates that the container should be used a source of Redis connection details. Spring Boot will provide built-in support for extracting those details from the container while still allowing the Testcontainers API to be used to define and configure the container.

[rajadilipkolli]

I have a use case where I am connecting to postgres using webflux but managing sql schema using liquibase, so in current process I need to set both database properties and Liquibase properties in @Dynamicpropertysource , will this update take care of this usecase as well?

[wilkinsona]

Yes. A single container will be able to provide multiple connections. Here's an example of a test that uses R2DBC with the database initialized using Liquibase (Flyway is also supported):

@DataR2dbcTest
@Testcontainers(disabledWithoutDocker = true)
class CityRepositoryTests {

    @Container
    @JdbcServiceConnection
    @R2dbcServiceConnection
    static PostgreSQLContainer<?> postgresql = new PostgreSQLContainer<>(DockerImageNames.postgresql())
        .withDatabaseName("test");

    @Autowired
    private CityRepository repository;

    @Test
    void databaseHasBeenInitialized() {
        StepVerifier.create(this.repository.findByState("DC").filter((city) -> city.getName().equals("Washington")))
            .consumeNextWith((city) -> assertThat(city.getId()).isNotNull())
            .verifyComplete();
    }

}

[joschi]

Does this feature/how does this feature relate to https://github.com/PlaytikaOSS/testcontainers-spring-boot?

[wilkinsona]

TIL about testcontainers-spring-boot… There's no relationship between the two.

[jucosorin]

Hi @wilkinsona

First of all, the new @ServiceConnection and related annotations are a great addition for running integration tests with Testcontainers. At the company I work for, I have created a Spring Boot Starter which is used for configuring Spring Kafka. The main autoconfiguration class kicks in with:

@AutoConfiguration(after = TaskSchedulingAutoConfiguration.class)
@EnableConfigurationProperties(StreamingKafkaProperties.class)
@ConditionalOnClass(name = "org.springframework.kafka.annotation.EnableKafka")
@ConditionalOnProperty(value = "mm.kafka.enabled", havingValue = "true")
@Import({
    StreamingKafkaSASLAutoConfiguration.class,
    StreamingKafkaProducerAutoConfiguration.class,
    StreamingKafkaConsumerAutoConfiguration.class,
    StreamingKafkaRetryAutoConfiguration.class,
    KafkaMessageService.class,
    AlertingMessageCreator.class,
    AlertingProducerService.class
})
public class StreamingKafkaAutoConfiguration {

We are doing integration tests using Testcontainers, and before this feature was introduced we were using the following to get the spring.kafka.bootstrap-servers injected into the Spring tests:

@Container
  private static final KafkaContainer kafkaContainer = new KafkaContainer(DockerImageName.parse(TESTCONTAINERS_KAFKA_IMAGE))
      .withEnv("KAFKA_AUTO_CREATE_TOPICS_ENABLE", "FALSE");

  @DynamicPropertySource
  static void setProperties(DynamicPropertyRegistry registry) {
    registry.add("spring.kafka.bootstrap-servers", kafkaContainer::getBootstrapServers);
  }

I wanted to switch to using @ServiceConnection and the code changed to:

@Container
@KafkaServiceConnection
private static final KafkaContainer kafkaContainer = new 
  KafkaContainer(DockerImageName.parse(TESTCONTAINERS_KAFKA_IMAGE))
      .withEnv("KAFKA_AUTO_CREATE_TOPICS_ENABLE", "FALSE");

The problem is that with this approach, the @KafkaServiceConnection test autoconfiguration kicks in after the code in my Spring Boot Starter and this results in "spring.kafka.bootstrap-servers" always being set to the default value of "PLAINTEXT://locahost:9092". The only solution to make this work is to add back the @DynamicPropertySource code since this injects the Testcontainers Kafka container port in my autoconfiguration classes.

Is there a more elegant solution to this? Am I not using this as it should be used? As I understand it, the whole point of using @KafkaServiceConnection is to not need to use the @DynamicPropertySource.

And btw, switching back to just using @DynamicPropertySource doesn't work either now because of https://github.com/spring-projects/spring-boot/issues/34770

[wilkinsona]

@jucosorin Thanks for trying out the milestone. Can you please open a separate issue for this?

[sergey-morenets]

Hi @wilkinsona

Are the code samples in this thread still relevant? I upgraded to Spring Boot 3.1.0 but couldn't find annotation @JdbcServiceConnection.

[eddumelendez]

Hi, it's only @ServiceConnection now. See https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#features.testing.testcontainers.service-connections

[sergey-morenets]

Hi @eddumelendez

Thank you for the quick response.

[pandamaroder]

Hi everyone and @wilkinsona, could you please help-

what a package name for this @RedisServiceConnection to import from?

[pandamaroder]

ive tried this

testImplementation("com.redis:testcontainers-redis:2.2.2")
implementation("org.springframework.boot:spring-boot-starter-data-redis")

testImplementation("org.springframework.boot:spring-boot-starter-test")

testImplementation("org.springframework.boot:spring-boot-testcontainers:2.6.0")
testImplementation("org.springframework.boot:spring-boot-test-autoconfigure:2.6.0")

[wilkinsona]

As we worked on this, we removed the need for service-specific annotations so there's no @RedisServiceConnection. You should use @ServiceConnection instead. You can learn more in the documentation.

[rajadilipkolli]

Hi @wilkinsona ,

For databases and others adding @ServiceConnection is enough but for using redis in SB 3.4.0-M1 , I still need to use @ServiceConnection("redis") . Is this done to support both redis and Redis-stack containers?

    @Bean
    @ServiceConnection("redis")
    RedisContainer redisContainer() {
        return new RedisContainer(DockerImageName.parse("redis").withTag("7.4.0-alpine"));
    }

[scottfrederick]

@rajadilipkolli There is an explanation of this in the documentation that Andy linked to above:

If you’re using a @Bean method, Spring Boot won’t call the bean method to get the Docker image name, because this would cause eager initialization issues. Instead, the return type of the bean method is used to find out which connection detail should be used. This works as long as you’re using typed containers, e.g. Neo4jContainer or RabbitMQContainer. This stops working if you’re using GenericContainer, e.g. with Redis, as shown in the following example:

What is the RedisContainer class that you are using? Is it an implementation of GenericContainer or something else?

[wilkinsona]

Could be related to https://github.com/spring-projects/spring-boot/issues/41450.

[rajadilipkolli]

yes, related to #41450, I am trying the milestone release in preparation for 3.4.0 release.