no way to map container port to different host port

[Bill]

docker -p 8003:5432 will expose container port 5432 on host port 8003.

This is extremely valuable when running parallel tests. It lets us avoid port conflicts.

I see no straightforward way to accomplish this with testcontainers.

Container.withExposedPorts(Integer... ports) takes a port array and maps each container port in the array, to the corresponding port on the host.

I expected to find a method with this signature something like this:

Container.withExposedPorts(Map<Integer,Integer> mappings)

Or maybe even:

Container.withExposedPortMappings(Integer... mappings)

…that would take the flattened mapping.

[bsideup]

It lets us avoid port conflicts.

TestContainers doesn't allow you (publically) to use a non-random port. When you call withExposedPorts(5432) it will use -p 5432, it means that the port will be random by default.

Then you just use getMappedPort(5432)

https://www.testcontainers.org/usage/generic_containers.html#accessing-a-container-from-tests

[Bill]

Thank you!!

[NickEm]

BTW, guys there is a possibility to specify host port as well. F.e:

int hostPort = 6380;
int containerExposedPort = 6379;
Consumer<CreateContainerCmd> cmd = e -> e.withPortBindings(new PortBinding(Ports.Binding.bindPort(hostPort), new ExposedPort(containerExposedPort)));

GenericContainer redisContainer = new GenericContainer("redis:4.0.10")
                    .withExposedPorts(containerExposedPort)
                    .withCreateContainerCmdModifier(cmd);

[ocker-docker-gummi-klokker]

import org.testcontainers.containers.GenericContainer import org.testcontainers.containers.FixedHostPortGenericContainer

Use GenericContainer container = new GenericContainer("name") when working with random ports.

Use GenericContainer container = new FixedHostPortGenericContainer("name") when working with fixed ports, and then do container.withFixedExposedPort(hostPort, containerPort)

[kiview]

While this works, we strongly advise against using fixed ports, since this will automatically lead to integrated tests (which are an anti pattern).

The definition of integrated tests based on this Spotify blog post:

A test that will pass or fail based on the correctness of another system.

[ocker-docker-gummi-klokker]

I understand and totally agree. All our automated tests use random ports, but we have a single local acceptance test (disabled in our CI environment) which for good reasons use fixed ports. In special situations, it is convenient to be able to use fixed ports.

[stahloss]

How do I bind the random host port to my @SpringBootTest Spring Data property? For instance: spring.data.mongodb.port=xxxxx.

Would be nice if I could do something like spring.data.mongodb.port=${testcontainers.host.port}.

Now I need to use the FixedHostPortGenericContainer, because I havent figured a way around this yet.

[bsideup]

@D0rmouse please look at our Spring Boot Example

[callamd]

While this works, we strongly advise against using fixed ports, since this will automatically lead to integrated tests (which are an anti pattern).

The definition of integrated tests based on this Spotify blog post:

A test that will pass or fail based on the correctness of another system.

Tenuous

[andrejpetras]

While this works, we strongly advise against using fixed ports, since this will automatically lead to integrated tests (which are an anti pattern).

The definition of integrated tests based on this Spotify blog post:

A test that will pass or fail based on the correctness of another system.

Could you please make the method withFixedExposedPort public in the GenericContainer? I believe the developers should have the option to decide what they need.

Currently we need to do workaround for example for the PostgreSQLContainer and we are building the library around the testcontainers.

[thergbway]

As far as I see there are some ways to create fixed port bindings for single containers but there is no any way to make fixed port binding for DockerComposeContainer. In our case we want a fixed exposed port for debug the java application we are testing. The target of out tests is one single service of started docker-compose environment. How to make fixed exposed ports for DockerComposeContainer?

[bsideup]

We do not recommend using fixed ports, even with single containers.

For debugging, consider using the approach described in the following blogpost: https://bsideup.github.io/posts/debugging_containers/

[bsideup]

@andrejpetras

even if we expose withFixedExposedPort it won't work in some environments (like running a build inside a container, or even Travis). It is protected for a good reason, and there are other ways of doing this (e.g. by proxying, see my previous comment on this issue)

[chinthaka-dinadasa]

BTW, guys there is a possibility to specify host port as well. F.e:

int hostPort = 6380;
int containerExposedPort = 6379;
Consumer<CreateContainerCmd> cmd = e -> e.withPortBindings(new PortBinding(Ports.Binding.bindPort(hostPort), new ExposedPort(containerExposedPort)));

GenericContainer redisContainer = new GenericContainer("redis:4.0.10")
                    .withExposedPorts(containerExposedPort)
                    .withCreateContainerCmdModifier(cmd);

Unfortunately withPortBindings Deprecated in newest version

[not-a-doctor-stromberg]

@D0rmouse please look at our Spring Boot Example

Maybe we could add this to the docs? https://github.com/testcontainers/testcontainers-java/pull/2484

[vietj]

It can be very useful to define a fixed port when you want to use a packet sniffer for debugging locally.

Unfortunately the method for defining a fixed mapping is protected. I could create a subclass to achieve fixed port mapping and be able to define a port I could sniff with Wireshark.

Here is what I did using an inner class:

  public static class FixedPostgreSQLContainer extends PostgreSQLContainer {
    public FixedPostgreSQLContainer(String dockerImageName) {
      super(dockerImageName);
    }
    public FixedPostgreSQLContainer configurePort() {
      super.addFixedExposedPort(5432, 5432);
      return this;
    }
  }

I'm posting this because it might be useful to others.

[bsideup]

@vietj please see https://bsideup.github.io/posts/testcontainers_fixed_ports/ and then https://bsideup.github.io/posts/debugging_containers/

The method is protected for reason (in fact, I really want to see it deprecated and removed soon /cc @rnorth )

[vietj]

I am merely saying it should be always available but not the default.

There should be the option to use it when it is useful. In the project I'm doing we are not using it by default but when we need to use Wireshark to understand what happens at the protocol level then we can simply replace the default container by the code I exhibited and be able to have access easily to the information we need.

So removing this "backdoor" would make life of developers like me harder.

[bsideup]

@vietj one can always modify the CreateContainerCmd and apply any (potentially dangerous) modification (s)he wants.

Just we don't want to have this API because it is known to be dangerous, especially for those who don't understand the problem and want to use it for something other than debugging (e.g. they hardcode localhost:5432 in their test app's config)

[bsideup]

@@vietj also, have you checked the second link? Is there any issue with using it with Wireshark or anything?

[vietj]

@bsideup I can't use a proxy in my tests because it is important to have the entire IP traffic between the hosts. Beyond observing what happens on the wire, I often need to check the all IP packets such as ACK or FINACK or RST between the two parties.

[Samehadar]

I had to use @vietj dirty hack today. I need this coz I use postgresql db in container as in-memory db for my app. And for querying/debugging data in my db I use IntelliJ Idea or DBeaver. Without this API I have to look for the containers port each time I start my app. Not that it is really important case, but I still don't wanna change my Data Source in IntelliJ Idea every time I start my application.

[jdeppe-pivotal]

I'd like you to consider NOT deprecating and removing the port binding API. Although I can understand you urging folks not to use it, I don't think you can always foresee all use cases and being overly strict doesn't end up helping :).

In my particular case I'm trying to build up a Redis cluster which should be externally accessible from the host system. Redis clustering has a command (CLUSTER SLOTS) which reports the IP and ports that members are accessible on. Typically this reports the internal docker IP and the same port for every system making up the cluster. Now, it's a chicken-n-egg problem if I want to use randomly assigned external ports which also need to be known when the cluster is started. Since there's no way for CLUSTER SLOTS to magically report externally exposed ports.

[Aside] To avoid the hardcoded port problem; in my case I'm going to use externally determined 'free' ports and use those when configuring the cluster.

In the case of Redis clustering there is a solution for Redis 6.0 (https://github.com/bitnami/bitnami-docker-redis-cluster/issues/7), however my point hopefully serves more as an example of a situation that would benefit from having the port binding API remain in place (undeprecated).

[bsideup]

@jdeppe-pivotal this sounds very similar to Kafka there we solved the problem by deferring starting the process inside a container. See https://github.com/testcontainers/testcontainers-java/blob/eaf9b9fe2aab4e60a4b0079e7c15afb38bb44beb/modules/kafka/src/main/java/org/testcontainers/containers/KafkaContainer.java#L106

Which only proves that there are valid options to the problem that do not require fixed ports.

[maxandersen]

I use testcontainer api to do quick'n'dirty experiments where fixed ports are good.

Being able to use testcontainers api to setup a set of containers on random ports is awesome; but equally awesome is it to set it up explicitly to fit into environment/usecase where static port is actually useful. i.e. for easy access to a database from external tools as already mentioned, but also for cases where the setup is hardwired in at a stage I can't (yet) change.

Big +1 for not removing the api.

[bsideup]

Note that it will always be possible to fix the ports because Docker supports it, and we allow modifying the underlying CreateContainerCmd. Just we won't offer a convenient API for that, as we find it dangerous.

Last but not least, one can use a proxy to guarantee that the host will be localhost (one of the biggest problem of fixed ports is not the ports themselves, but the fact that users hardcode the host, and, unlike ports, host isn't always static): https://bsideup.github.io/posts/debugging_containers/

[desiderati]

BTW, guys there is a possibility to specify host port as well. F.e:

int hostPort = 6380;
int containerExposedPort = 6379;
Consumer<CreateContainerCmd> cmd = e -> e.withPortBindings(new PortBinding(Ports.Binding.bindPort(hostPort), new ExposedPort(containerExposedPort)));

GenericContainer redisContainer = new GenericContainer("redis:4.0.10")
                    .withExposedPorts(containerExposedPort)
                    .withCreateContainerCmdModifier(cmd);

Becuase this method (withPortBindings) is deprecated, you can do it now like this:

static final MySQLContainer<?> mysql =
    new MySQLContainer<>("mysql:5.6")
        .withExposedPorts(34343)
        .withCreateContainerCmdModifier(cmd -> cmd.withHostConfig(
            new HostConfig().withPortBindings(new PortBinding(Ports.Binding.bindPort(34343), new ExposedPort(3306)))
        ));

[mklueh]

BTW you need to cast to Consumer if you are using the lambda way of implementation

@Container
@SuppressWarnings({"rawtypes", "unchecked"}) //otherwise everything yellow in the IDE
private final GenericContainer sftp = new GenericContainer("atmoz/sftp")
        .withExposedPorts(PORT)
        .withCreateContainerCmdModifier((Consumer<CreateContainerCmd>) cmd -> cmd.withHostConfig(
                new HostConfig().withPortBindings(new PortBinding(Ports.Binding.bindPort(PORT), new ExposedPort(22222)))
        ));

[bsideup]

@mklueh Or like this:

@Container
private final GenericContainer<?> sftp = new GenericContainer<>("atmoz/sftp")
        .withExposedPorts(PORT)
        .withCreateContainerCmdModifier(cmd -> cmd.withHostConfig(
                new HostConfig().withPortBindings(new PortBinding(Ports.Binding.bindPort(PORT), new ExposedPort(22222)))
        ));

[typik89]

It works for me. companion object{ @Container val postgres = PostgreSQLContainer<Nothing>("postgres:13-alpine") .apply { portBindings.add("5432:5432") } }

In java: postgres.getPortBindings().add("5432:5432")

[fresh-fx59]

May be I am doing something wrong, but while launching kafka in testcontainers in kraft mode I don't understand how can I set specific port before start rather than make it fixed.

Its ok to connect to kafka with localhost:RANDOM_PORT, but in ENV there is hardcoded port that returns in metadata and app tries to use IT instead. So the problem is that I can connect with random port, but can't add it in settings before container starts.

@typik89 solution was exceptionally great. This solution doesn't work for me.

I've taken port chooser from here

import lombok.extern.slf4j.Slf4j;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.testcontainers.containers.BindMode;
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.containers.PostgreSQLContainer;
import org.testcontainers.containers.output.Slf4jLogConsumer;
import org.testcontainers.containers.wait.strategy.LogMessageWaitStrategy;
import org.testcontainers.utility.DockerImageName;
import mypackage.TestSocketUtils;

import java.time.Duration;
import java.util.Map;

/**
 *
 * https://rieckpil.de/reuse-containers-with-testcontainers-for-fast-integration-tests/
 */
@Slf4j
public abstract class Containers {

    private static final Integer KAFKA_PORT = TestSocketUtils.findAvailableTcpPort();
    private static final String KAFKA_IMAGE_NAME = "bitnami/kafka:3.3.1";

    // kafka

    public static final GenericContainer<?> kafkaContainer;

    static {

        final Map<String, String> kafkaSettings = Map.ofEntries(
                Map.entry("KAFKA_ENABLE_KRAFT", "yes"),
                Map.entry("KAFKA_CFG_PROCESS_ROLES", "broker,controller"),
                Map.entry("KAFKA_CFG_CONTROLLER_LISTENER_NAMES", "CONTROLLER"),
                Map.entry("KAFKA_CFG_LISTENERS", "PLAINTEXT://:" + KAFKA_PORT + ",CONTROLLER://:9093,CLIENT://:29092"),
                Map.entry("KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP", "CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,CLIENT:PLAINTEXT"),
                Map.entry("KAFKA_CFG_CONTROLLER_QUORUM_VOTERS", "1@127.0.0.1:9093"),
                Map.entry("KAFKA_CFG_ADVERTISED_LISTENERS", "PLAINTEXT://127.0.0.1:" + KAFKA_PORT + ",CLIENT://kafka:29092"),
                Map.entry("KAFKA_CFG_INTER_BROKER_LISTENER_NAME", "CLIENT"),
                Map.entry("KAFKA_CFG_BROKER_ID", "1"),
                Map.entry("ALLOW_PLAINTEXT_LISTENER", "yes"),
                Map.entry("KAFKA_KRAFT_CLUSTER_ID", "7rrriwLUQ4ydpNAMkl5PKA"),
                Map.entry("KAFKA_TOPIC_MAX_MESSAGE_BYTES", "30000000"),
                Map.entry("KAFKA_REPLICA_FETCH_MAX_BYTES", "30000000"),
                Map.entry("KAFKA_MESSAGE_MAX_BYTES", "30000000"),
                Map.entry("KAFKA_SOCKET_REQUEST_MAX_BYTES", "30000120"),
                Map.entry("TZ", "Europe/Moscow")
        );

        kafkaContainer = new GenericContainer<>(KAFKA_IMAGE_NAME)
                .withExposedPorts(KAFKA_PORT)
                .withEnv(kafkaSettings)
                .withLogConsumer(new Slf4jLogConsumer(log).withPrefix("tstcntrs.KAFKA"))
                .waitingFor(new LogMessageWaitStrategy()
                        .withRegEx(".*Kafka Server started.*\\s")
                        .withTimes(1)
                        .withStartupTimeout(Duration.ofMinutes(2)))
        ;

        kafkaContainer.getPortBindings().add(KAFKA_PORT + ":" + KAFKA_PORT);

        kafkaContainer.start();
    }

    @DynamicPropertySource
    static void datasourceConfig(DynamicPropertyRegistry registry) {
        // kafka
        registry.add("kafka.bootstrapAddress", () -> "127.0.0.1:" + KAFKA_PORT);
    }
}

[patraanjan23]

I faced the same issue. The cleanest solution I could come up with without using any deprecated methods is the following:

int HOST_PORT = 1234;
int CONTAINER_PORT = 2468;

PortBinding portBinding = new PortBinding(Ports.Binding.bindPort(HOST_PORT), new ExposedPort(CONTAINER_PORT));

public static GenericContainer<?> container = new GenericContainer<>(...)
    .withCreateContainerCmdModifier(cmd -> cmd.withHostConfig(new HostConfig().withPortBindings(portBinding))
    .withExposedPorts(CONTAINER_PORT)
    ...

[Zialus]

I faced the same issue. The cleanest solution I could come up with without using any deprecated methods is the following:

int HOST_PORT = 1234;
int CONTAINER_PORT = 2468;

PortBinding portBinding = new PortBinding(Ports.Binding.bindPort(HOST_PORT), new ExposedPort(CONTAINER_PORT));

public static GenericContainer<?> container = new GenericContainer<>(...)
    .withCreateContainerCmdModifier(cmd -> cmd.withHostConfig(new HostConfig().withPortBindings(portBinding))
    .withExposedPorts(CONTAINER_PORT)
    ...

for future reference, the api was changed from:

PortBinding portBinding = new PortBinding(Ports.Binding.bindPort(HOST_PORT), new ExposedPort(CONTAINER_PORT));

to:

Ports portBinding = new Ports(new ExposedPort(CONTAINER_PORT), Ports.Binding.bindPort(HOST_PORT));

[jamesdh]

for future reference, the api was changed from:

PortBinding portBinding = new PortBinding(Ports.Binding.bindPort(HOST_PORT), new ExposedPort(CONTAINER_PORT));

to:

Ports portBinding = new Ports(new ExposedPort(CONTAINER_PORT), Ports.Binding.bindPort(HOST_PORT));

@Zialus I'm not sure that's correct. TBF I'm not sure it makes any difference, but this is what the API doc for Ports states:

This is an abstraction used for querying existing port bindings from a container configuration. It is not to be confused with the PortBinding abstraction used for adding new port bindings to a container.

[Zialus]

for future reference, the api was changed from:

PortBinding portBinding = new PortBinding(Ports.Binding.bindPort(HOST_PORT), new ExposedPort(CONTAINER_PORT));

to:

Ports portBinding = new Ports(new ExposedPort(CONTAINER_PORT), Ports.Binding.bindPort(HOST_PORT));

@Zialus I'm not sure that's correct. TBF I'm not sure it makes any difference, but this is what the API doc for Ports states:

This is an abstraction used for querying existing port bindings from a container configuration. It is not to be confused with the PortBinding abstraction used for adding new port bindings to a container.

The question here is in regards to what withPortBindings is able to receive as an input

[jroper]

There is a valid use case for making this convenient that doesn't involve using fixed ports. The issue arises when the container needs to know the mapped port. In my case, I'm testing an openid client that I want to test, so I have a small openid provider that runs in a container to test it against. OpenID depends heavily on absolute URLs - you configure an openid provider to have a particular issuer URL. In my case, I'm using the node openid-provider, and it needs to know its absolute issuer URL at startup, so I need to know that mapped port before I start the container.

So, to do this without using a fixed port, I simply select a random port ephemeral before starting the container. This can be done by opening a listening socket on port 0, reading the ephemeral port number, then immediately closing it. Then I have a random free port, I'm not violating any testing bad practices, and I can pass that free port to my openid-provider container, and then use it when creating the port mapping.