testcontainers / testcontainers-python

Testcontainers is a Python library that providing a friendly API to run Docker container. It is designed to create runtime environment to use during your automatic tests.
https://testcontainers-python.readthedocs.io/en/latest/
Apache License 2.0
1.56k stars 281 forks source link

Bug: in v4 `get_exposed_port` not working with IPv6 localhost (on MacOS, colima) #553

Open ovidiu-munteanu opened 5 months ago

ovidiu-munteanu commented 5 months ago

Trying to migrate from testcontainers-python v3.7.1 to v4.4.0 and get_exposed_port isn't working as expected.

Using the construct exposed_port = engine.get_exposed_port(engine.port_to_expose) works fine in v3.7.1, giving the exposed port that can be used to access the DB. In v4.4.0 replacing this with exposed_port = engine.get_exposed_port(engine.port) doesn't work.

When running the tests I get the error below; the port number changes on every execution.

sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) connection to server at "localhost" (::1), port 33373 failed: Connection refused

Full existing code for v3.7.1:

@pytest.fixture(scope="session", name="pg_cluster")
def postgres_cluster() -> PostgresContainer:
    with PostgresContainer(
        PG_BASE_IMAGE, dbname="postgres", user="postgres", password="postgres"
    ) as postgres:
        # run migrations
        alembic_config = AlembicConfig("alembic.ini")
        alembic_config.set_main_option("sqlalchemy.url", postgres.get_connection_url())
        upgrade(alembic_config, "head")
        yield postgres

@pytest.fixture(scope="session")
def application_config(pg_cluster: PostgresContainer) -> Iterator[ApplicationConfig]:
    config = load_application_config()
    update_host = {
        "hosts": [pg_cluster.get_container_host_ip()],
        "port": pg_cluster.get_exposed_port(pg_cluster.port_to_expose),
    }

    update_database = {
        "ddl_database": config.ddl_database.copy(update=update_host),
        "database": config.database.copy(update=update_host),
    }
    new_config = config.copy(update=update_database)
    yield new_config

Full updated code for v4.4.0:

@pytest.fixture(scope="session", name="pg_cluster")
def postgres_cluster() -> PostgresContainer:
    with PostgresContainer(
        PG_BASE_IMAGE, dbname="postgres", username="postgres", password="postgres"
    ) as postgres:
        # run migrations
        alembic_config = AlembicConfig("alembic.ini")
        alembic_config.set_main_option("sqlalchemy.url", postgres.get_connection_url())
        upgrade(alembic_config, "head")
        yield postgres

@pytest.fixture(scope="session")
def application_config(pg_cluster: PostgresContainer) -> Iterator[ApplicationConfig]:
    config = load_application_config()
    update_host = {
        "hosts": [pg_cluster.get_container_host_ip()],
        "port": pg_cluster.get_exposed_port(pg_cluster.port),
    }

    update_database = {
        "ddl_database": config.ddl_database.copy(update=update_host),
        "database": config.database.copy(update=update_host),
    }
    new_config = config.copy(update=update_database)
    yield new_config

Runtime environment

Python 3.11.9

Darwin LDN-XXXXXXXXXX 23.4.0 Darwin Kernel Version 23.4.0: Fri Mar 15 00:10:42 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_T6000 arm64

Docker Info ``` Client: Docker Engine - Community Version: 26.0.1 Context: default Debug Mode: false Plugins: compose: Docker Compose (Docker Inc.) Version: 2.26.1 Path: /Users/user.name/.docker/cli-plugins/docker-compose Server: Containers: 0 Running: 0 Paused: 0 Stopped: 0 Images: 2 Server Version: 24.0.9 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Using metacopy: false Native Overlay Diff: true userxattr: false Logging Driver: json-file Cgroup Driver: cgroupfs Cgroup Version: 2 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 runc Default Runtime: runc Init Binary: docker-init containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb runc version: v1.1.12-0-g51d5e94 init version: de40ad0 Security Options: apparmor seccomp Profile: builtin cgroupns Kernel Version: 6.5.0-15-generic Operating System: Ubuntu 23.10 OSType: linux Architecture: aarch64 CPUs: 4 Total Memory: 5.775GiB Name: colima ID: ff9f7b4d-3e2a-426b-abc7-6c26ea211a18 Docker Root Dir: /var/lib/docker Debug Mode: false Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false ``` Pip ``` aenum==3.1.15 aiodns==3.2.0 aiohttp==3.9.5 aioresponses==0.7.6 aiosignal==1.3.1 alembic==1.13.1 alembic_utils==0.8.2 aniso8601==9.0.1 anyio==3.7.1 astroid==2.15.8 asttokens==2.4.1 attrs==23.2.0 Babel==2.14.0 beautifulsoup4==4.12.3 black==23.12.1 blinker==1.7.0 Brotli==1.1.0 cachetools==5.3.3 certifi==2024.2.2 cffi==1.16.0 charset-normalizer==3.3.2 click==8.1.7 cloudpickle==3.0.0 ConfigArgParse==1.7 contourpy==1.2.1 countryguess==0.3.0 coverage==7.4.4 cryptography==42.0.5 cycler==0.12.1 datasets==2.19.0 decorator==5.1.1 Deprecated==1.2.14 deprecation==2.1.0 deptry==0.16.1 dill==0.3.8 distro==1.9.0 docker==7.0.0 elastic-transport==8.13.0 elasticsearch==8.13.0 entrypoints==0.4 evaluate==0.4.1 execnet==2.1.1 executing==2.0.1 factory-boy==3.3.0 Faker==19.13.0 fastapi==0.109.2 filelock==3.13.4 flake8==7.0.0 Flake8-pyproject==1.2.3 Flask==3.0.3 Flask-Cors==4.0.0 Flask-Login==0.6.3 flupy==1.2.0 fonttools==4.51.0 frozenlist==1.4.1 fsspec==2024.3.1 gevent==24.2.1 geventhttpclient==2.2.1 gitdb==4.0.11 GitPython==3.1.41 google-api-core==2.18.0 google-auth==2.29.0 google-cloud-core==2.4.1 google-cloud-storage==2.16.0 google-cloud-translate==3.15.3 google-crc32c==1.5.0 google-resumable-media==2.7.0 googleapis-common-protos==1.63.0 graphene==3.3 graphql-core==3.2.3 graphql-relay==3.2.0 greenlet==3.0.3 grpcio==1.62.2 grpcio-status==1.62.2 gunicorn==20.1.0 h11==0.14.0 html2text==2020.1.16 httpcore==0.16.3 httpx==0.23.3 huggingface-hub==0.22.2 hvac==0.11.2 idave==5.7.1 idna==3.7 importlib_metadata==7.1.0 iniconfig==2.0.0 ipython==8.23.0 isort==5.13.2 itsdangerous==2.2.0 jedi==0.19.1 Jinja2==3.1.3 joblib==1.4.0 jsonpointer==2.4 kiwisolver==1.4.5 langcodes==3.3.0 langdetect==1.0.9 lazy-object-proxy==1.10.0 locust==2.26.0 Mako==1.3.3 Markdown==3.6 markdownmaker==0.4.0 marko==2.0.3 MarkupSafe==2.1.5 matplotlib==3.8.4 matplotlib-inline==0.1.7 mccabe==0.7.0 mlflow==2.12.1 more-itertools==9.1.0 msgpack==1.0.8 multidict==6.0.5 multiprocess==0.70.16 mypy==1.9.0 mypy-extensions==1.0.0 newrelic==9.9.0 numpy==1.26.4 openai==1.23.2 orjson==3.10.1 packaging==23.2 pandas==2.2.2 parse==1.20.1 parso==0.8.4 pathspec==0.12.1 patsy==0.5.6 pexpect==4.9.0 pillow==10.3.0 platformdirs==4.2.0 pluggy==1.5.0 prompt-toolkit==3.0.43 proto-plus==1.23.0 protobuf==4.25.3 psutil==5.9.8 psycopg2==2.9.9 ptyprocess==0.7.0 pure-eval==0.2.2 pyarrow==15.0.2 pyarrow-hotfix==0.6 pyasn1==0.6.0 pyasn1_modules==0.4.0 pycares==4.4.0 pycodestyle==2.11.1 pycparser==2.22 pydantic==1.10.15 pyflakes==3.2.0 Pygments==2.17.2 pyjnius==1.5.0 PyJWT==2.8.0 pylint==2.17.7 pyparsing==3.1.2 pytest==7.4.4 pytest-aioresponses==0.2.0 pytest-asyncio==0.23.6 pytest-cov==5.0.0 pytest-mock==3.14.0 pytest-xdist==3.5.0 python-dateutil==2.9.0.post0 python-dotenv==1.0.1 pytz==2024.1 PyYAML==6.0.1 pyzmq==26.0.2 querystring-parser==1.2.4 regex==2024.4.16 requests==2.31.0 requests-mock==1.12.1 responses==0.18.0 respx==0.21.1 rfc3986==1.5.0 roundrobin==0.0.4 rsa==4.9 scikit-learn==1.4.2 scipy==1.13.0 six==1.16.0 smmap==5.0.1 sniffio==1.3.1 soupsieve==2.5 SQLAlchemy==2.0.29 sqlparse==0.5.0 stack-data==0.6.3 starlette==0.36.3 statsmodels==0.14.2 stringcase==1.2.0 teamcity-messages==1.32 testcontainers==3.7.1 threadpoolctl==3.4.0 tiktoken==0.5.2 time-machine==2.14.1 toml==0.10.2 tomli==2.0.1 tomlkit==0.12.4 tqdm==4.66.2 traitlets==5.14.3 trino==0.327.0 types-beautifulsoup4==4.12.0.20240229 types-html5lib==1.1.11.20240228 types-Markdown==3.6.0.20240316 types-psycopg2==2.9.21.20240417 types-PyYAML==6.0.12.20240311 types-requests==2.31.0.20240406 types-toml==0.10.8.20240310 types-urllib3==1.26.25.14 typing_extensions==4.11.0 tzdata==2024.1 tzlocal==5.2 urllib3==2.2.1 uvicorn==0.20.0 uvloop==0.17.0 wcwidth==0.2.13 Werkzeug==3.0.2 workflow-client==1.2.3 wrapt==1.16.0 xxhash==3.4.1 yarl==1.9.4 zipp==3.18.1 zope.event==5.0 zope.interface==6.3 ```
alexanderankin commented 5 months ago

we recently added a delay for colima, but im not a colima user so i am relying on #543 / #486 here for context

alexanderankin commented 5 months ago

oh that is including that fix. okay, so this is probably unrelated, maybe an ipv6 having different ports and we just grab the first one.