Closed fabricebaranski closed 3 weeks ago
Moreover, I have some difficulties to create communication between these two containers on gitlab ci.
I also experience some difficulties with advanced network setup and I came to the conclusion it's due to the way it's implemented in TC:
self._container = docker_client.run
is executed without network
kwarg, this means container has default network assigned by default:
if "network" not in kwargs and not get_docker_host():
# Otherwise we'll try to find the docker host for dind usage.
host_network = self.find_host_network()
if host_network:
kwargs["network"] = host_network
self._network
is not empty (after adding .with_network
to container definition), container is connected to it (so it now is connected to two networks):
if self._network:
self._network.connect(self._container.id, self._network_aliases)
the solution could be to pass network
kwarg to container constructor and omit .with_network
completely but then we cannot assign network aliases. The workaround for this scenario is to use .with_name
(it will be used as network alias as well) - I usually generate it with some random characters inside to avoid name clashes. Example:
network = Network()
network.create()
MYSQL_CONTAINER_NAME = "mysql-" + generate_random_string_of_length(8)
mysql_container = (
MySqlContainer(
f"mysql:{MYSQL_IMAGE_TAG}",
username=MYSQL_USER,
password=MYSQL_PASSWORD,
root_password=MYSQL_ROOT_PASSWORD,
dbname=MYSQL_DB,
network=network.name
)
.with_name(MYSQL_CONTAINER_NAME)
.with_command("--sort_buffer_size=10M")
)
It seems to work when I add .with_bind_ports(9099, 0)
Confirming this does not work, for good reasons:
When calling docker_client.run
in container.py#start, it doesn't pass the network through.
If you look at how the method behaves, it will then resort to host networking which is not meant to support port bindings. : if you run docker port
on the created container, no port will show up.
By the time the network is connected (L102), it is too late.
Instead, this is what start should look like:
....
self._container = docker_client.run(
self.image,
command=self._command,
detach=True,
environment=self.env,
ports=self.ports,
name=self._name,
volumes=self.volumes,
# New params
network=network.name,
networking_config={network.name: EndpointConfig(version, aliases=self._network_aliases, **self._network_endpoint_configs) )
**self._kwargs,
)
In the meantime a workaround is:
# replace:
# ctr.with_network(network)
# ctr.with_network_aliases("network_alias")
# with:
ctr.with_kwargs(
network=network.name, networking_config={network.name: EndpointConfig("1.33", aliases=["network_alias"])}
)
Also adding a quick reproducer for testing:
def kafka(network: Network):
ctr = KafkaContainer(image=f"confluentinc/cp-kafka:7.6.1")
ctr.with_network(network)
ctr.with_network_aliases("kafka")
# uncomment to fix
# ctr.with_kwargs(network=network.name, networking_config={network.name: EndpointConfig("1.33", aliases=["kafka"])})
with ctr:
assert re.match("[^/:]+:[0-9]{4,5}", ctr.get_bootstrap_server()).group()
this is what start should look like
open to a PR for this
I struggled for hours with the same problem, and got this to work (in Pytest's fixtures) :
MQTT_BROKER_NETWORK_ALIAS = "mqtt_broker"
@pytest.fixture()
def network_for_test() -> Generator[Network, None, None]:
test_network = Network()
with test_network:
yield test_network
@pytest.fixture()
def mqtt_broker(network_for_test: Network) -> Generator[MosquittoContainer, None, None]:
mqtt_broker = MosquittoContainer() \
.with_network(network_for_test) \
.with_network_aliases(MQTT_BROKER_NETWORK_ALIAS)
with mqtt_broker:
yield mqtt_broker
@pytest.fixture()
def web_server(mqtt_broker: MosquittoContainer,
network_for_test: Network) -> Generator[ServerContainer, None, None]:
# [...]
web_server = ServerContainer(
image="something", port=80
).with_network(network_for_test)
# [...]
with web_server:
yield web_server
This way, I can make the web server connect to the MQTT broker (using the MQTT_BROKER_NETWORK_ALIAS
).
I've combined @champialex and mine findings into https://github.com/testcontainers/testcontainers-python/pull/678
Hi all, I would like to have two containers connected each others, with a code similar to
I regularly have no port mapping for my containers. Do you have any ideas?