search

testcontainers / testcontainers-python

Testcontainers is a Python library that providing a friendly API to run Docker container. It is designed to create runtime environment to use during your automatic tests.
https://testcontainers-python.readthedocs.io/en/latest/
Apache License 2.0
1.51k stars 281 forks source link

Bug: Docker Compose test is broken #570

Closed Tranquility2 closed 3 months ago

Tranquility2 commented 3 months ago

Describe the bug

It seems that with the latest version of docker compose installed the test_compose_logs fails

To Reproduce

  1. Update docker (Linux machine) *My guess is docker-compose-plugin
  2. Pull main branch of testcontainers-python
  3. Run the tests Can be as simple as 
    pytest core/tests/test_compose.py

Example 

=================================================== test session starts ====================================================
platform linux -- Python 3.9.18, pytest-7.4.3, pluggy-1.4.0
rootdir: /home/roy/projects/testcontainers-python
configfile: pyproject.toml
plugins: anyio-4.3.0, asyncio-0.23.5, cov-4.1.0
asyncio: mode=strict
collected 11 items

core/tests/test_compose.py::test_compose_no_file_name PASSED                                                         [  9%]
core/tests/test_compose.py::test_compose_str_file_name PASSED                                                        [ 18%]
core/tests/test_compose.py::test_compose_list_file_name PASSED                                                       [ 27%]
core/tests/test_compose.py::test_compose_stop PASSED                                                                 [ 36%]
core/tests/test_compose.py::test_compose_start_stop PASSED                                                           [ 45%]
core/tests/test_compose.py::test_compose PASSED                                                                      [ 54%]
core/tests/test_compose.py::test_compose_logs FAILED                                                                 [ 63%]
core/tests/test_compose.py::test_compose_ports
------------------------------------------------------ live log call -------------------------------------------------------
INFO     testcontainers.core.waiting_utils:waiting_utils.py:54 Waiting for DockerCompose(context=PosixPath('/home/roy/projects/testcontainers-python/core/tests/compose_fixtures/port_single'), compose_file_name=None, pull=False, build=False, wait=True, env_file=None, services=None, docker_command_path=None) to be ready ...
PASSED                                                                                                               [ 72%]
core/tests/test_compose.py::test_compose_multiple_containers_and_ports PASSED                                        [ 81%]
core/tests/test_compose.py::test_exec_in_container
------------------------------------------------------ live log call -------------------------------------------------------
INFO     testcontainers.core.waiting_utils:waiting_utils.py:54 Waiting for DockerCompose(context=PosixPath('/home/roy/projects/testcontainers-python/core/tests/compose_fixtures/port_single'), compose_file_name=None, pull=False, build=False, wait=True, env_file=None, services=None, docker_command_path=None) to be ready ...
PASSED                                                                                                               [ 90%]
core/tests/test_compose.py::test_exec_in_container_multiple
------------------------------------------------------ live log call -------------------------------------------------------
INFO     testcontainers.core.waiting_utils:waiting_utils.py:54 Waiting for DockerCompose(context=PosixPath('/home/roy/projects/testcontainers-python/core/tests/compose_fixtures/port_multiple'), compose_file_name=None, pull=False, build=False, wait=True, env_file=None, services=None, docker_command_path=None) to be ready ...
PASSED                                                                                                               [100%]

========================================================= FAILURES =========================================================
____________________________________________________ test_compose_logs _____________________________________________________
core/tests/test_compose.py:89: in test_compose_logs
    assert not stderr
E   assert not 'time="2024-05-12T19:50:00Z" level=warning msg="/home/roy/projects/testcontainers-python/core/tests/compose_fixtures/basic/docker-compose.yaml: `version` is obsolete"\n'
================================================= short test summary info ==================================================
FAILED core/tests/test_compose.py::test_compose_logs - assert not 'time="2024-05-12T19:50:00Z" level=warning msg="/home/roy/projects/testcontainers-python/core/tests/compose_...
============================================== 1 failed, 10 passed in 17.75s ===============================================

Runtime environment

Server: Containers: 3 Running: 1 Paused: 0 Stopped: 2 Images: 7 Server Version: 25.0.4 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Using metacopy: false Native Overlay Diff: false userxattr: true Logging Driver: json-file Cgroup Driver: systemd Cgroup Version: 2 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 runc Default Runtime: runc Init Binary: docker-init containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb runc version: v1.1.12-0-g51d5e94 init version: de40ad0 Security Options: seccomp Profile: builtin rootless cgroupns Kernel Version: 5.15.0-100-generic Operating System: Ubuntu 22.04.4 LTS OSType: linux Architecture: x86_64 CPUs: 2 Total Memory: 7.752GiB Name: bow ID: 76ee101e-0272-4884-925b-467f3395fa5f Docker Root Dir: /home/roy/.local/share/docker Debug Mode: false Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false

WARNING: No cpu cfs quota support WARNING: No cpu cfs period support WARNING: No cpu shares support WARNING: No cpuset support WARNING: No io.weight support WARNING: No io.weight (per device) support WARNING: No io.max (rbps) support WARNING: No io.max (wbps) support WARNING: No io.max (riops) support WARNING: No io.max (wiops) support

alexanderankin commented 3 months ago

Yep this is fixed in the typing pr but it was super ceded by more urgent pr's so it never got fixed, pr is welcome

On Sun, May 12, 2024, 3:51 PM Roy Moore @.***> wrote:

Describe the bug

It seems that with the latest version of docker compose installed the test_compose_logs fails

To Reproduce

  1. Update docker (Linux machine) *My guess is docker-compose-plugin
  2. Pull main branch of testcontainers-python
  3. Run the tests Can be as simple as

pytest core/tests/test_compose.py

Example

=================================================== test session starts ==================================================== platform linux -- Python 3.9.18, pytest-7.4.3, pluggy-1.4.0 rootdir: /home/roy/projects/testcontainers-python configfile: pyproject.toml plugins: anyio-4.3.0, asyncio-0.23.5, cov-4.1.0 asyncio: mode=strict collected 11 items

core/tests/test_compose.py::test_compose_no_file_name PASSED [ 9%] core/tests/test_compose.py::test_compose_str_file_name PASSED [ 18%] core/tests/test_compose.py::test_compose_list_file_name PASSED [ 27%] core/tests/test_compose.py::test_compose_stop PASSED [ 36%] core/tests/test_compose.py::test_compose_start_stop PASSED [ 45%] core/tests/test_compose.py::test_compose PASSED [ 54%] core/tests/test_compose.py::test_compose_logs FAILED [ 63%] core/tests/test_compose.py::test_compose_ports ------------------------------------------------------ live log call ------------------------------------------------------- INFO testcontainers.core.waiting_utils:waiting_utils.py:54 Waiting for DockerCompose(context=PosixPath('/home/roy/projects/testcontainers-python/core/tests/compose_fixtures/port_single'), compose_file_name=None, pull=False, build=False, wait=True, env_file=None, services=None, docker_command_path=None) to be ready ... PASSED [ 72%] core/tests/test_compose.py::test_compose_multiple_containers_and_ports PASSED [ 81%] core/tests/test_compose.py::test_exec_in_container ------------------------------------------------------ live log call ------------------------------------------------------- INFO testcontainers.core.waiting_utils:waiting_utils.py:54 Waiting for DockerCompose(context=PosixPath('/home/roy/projects/testcontainers-python/core/tests/compose_fixtures/port_single'), compose_file_name=None, pull=False, build=False, wait=True, env_file=None, services=None, docker_command_path=None) to be ready ... PASSED [ 90%] core/tests/test_compose.py::test_exec_in_container_multiple ------------------------------------------------------ live log call ------------------------------------------------------- INFO testcontainers.core.waiting_utils:waiting_utils.py:54 Waiting for DockerCompose(context=PosixPath('/home/roy/projects/testcontainers-python/core/tests/compose_fixtures/port_multiple'), compose_file_name=None, pull=False, build=False, wait=True, env_file=None, services=None, docker_command_path=None) to be ready ... PASSED [100%]

========================================================= FAILURES ========================================================= ____ test_composelogs ____ core/tests/test_compose.py:89: in test_compose_logs assert not stderr E assert not 'time="2024-05-12T19:50:00Z" level=warning msg="/home/roy/projects/testcontainers-python/core/tests/compose_fixtures/basic/docker-compose.yaml: version is obsolete"\n' ================================================= short test summary info ================================================== FAILED core/tests/test_compose.py::test_composelogs - assert not 'time="2024-05-12T19:50:00Z" level=warning msg="/home/roy/projects/testcontainers-python/core/tests/compose...============================================== 1 failed, 10 passed in 17.75s ===============================================

Runtime environment

  • Linux (Ubuntu) Linux bow 5.15.0-100-generic #110-Ubuntu SMP Wed Feb 7 13:27:48 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
  • Python 3.9 Python 3.9.18

alabaster==0.7.16 annotated-types==0.6.0 anyio==4.3.0 argon2-cffi==23.1.0 argon2-cffi-bindings==21.2.0 asn1crypto==1.5.1 asttokens==2.4.1 async-timeout==4.0.3 attrs==23.2.0 Authlib==1.3.0 azure-core==1.30.1 azure-storage-blob==12.19.1 Babel==2.14.0 backoff==2.2.1 bcrypt==4.1.2 black==24.4.2 boto3==1.34.59 botocore==1.34.59 cachetools==5.3.3 cassandra-driver==3.29.1 certifi==2024.2.2 cffi==1.16.0 cfgv==3.4.0 charset-normalizer==3.3.2 chromadb-client==0.4.25.dev0 click==8.1.7 clickhouse-driver==0.2.7 coverage==7.4.3 cryptography==42.0.5 decorator==5.1.1 Deprecated==1.2.14 deprecation==2.1.0 distlib==0.3.8 dnspython==2.6.1 docker==7.0.0 docutils==0.20.1 exceptiongroup==1.2.0 executing==2.0.1 filelock==3.13.1 geomet==0.2.1.post1 google-api-core==2.17.1 google-auth==2.28.2 google-cloud-core==2.4.1 google-cloud-datastore==2.19.0 google-cloud-pubsub==2.20.1 googleapis-common-protos==1.62.0 greenlet==3.0.3 grpc-google-iam-v1==0.13.0 grpcio==1.62.1 grpcio-health-checking==1.62.1 grpcio-status==1.62.1 grpcio-tools==1.62.1 h11==0.14.0 h2==4.1.0 hpack==4.0.0 httpcore==1.0.5 httpx==0.27.0 hvac==2.1.0 hyperframe==6.0.1 identify==2.5.35 idna==3.6 imagesize==1.4.1 importlib_metadata==7.0.2 influxdb==5.3.1 influxdb-client==1.41.0 iniconfig==2.0.0 ipython==8.18.1 isodate==0.6.1 jaraco.classes==3.3.1 jedi==0.19.1 jeepney==0.8.0 Jinja2==3.1.3 jmespath==1.0.1 jwcrypto==1.5.6 kafka-python-ng==2.2.0 keyring==24.3.1 kubernetes==29.0.0 markdown-it-py==3.0.0 MarkupSafe==2.1.5 matplotlib-inline==0.1.7 mdurl==0.1.2 minio==7.2.5 monotonic==1.6 more-itertools==10.2.0 msgpack==1.0.8 mypy==1.7.1 mypy-extensions==1.0.0 nats-py==2.7.2 neo4j==5.18.0 nh3==0.2.15 nodeenv==1.8.0 numpy==1.26.4 oauthlib==3.2.2 opensearch-py==2.4.2 opentelemetry-api==1.16.0 opentelemetry-exporter-otlp-proto-grpc==1.16.0 opentelemetry-proto==1.16.0 opentelemetry-sdk==1.16.0 opentelemetry-semantic-conventions==0.37b0 oracledb==2.1.1 orjson==3.10.0 outcome==1.3.0.post0 overrides==7.7.0 packaging==24.0 parso==0.8.4 pathspec==0.12.1 pexpect==4.9.0 pg8000==1.30.5 pika==1.3.2 pkginfo==1.10.0 platformdirs==4.2.0 pluggy==1.4.0 portalocker==2.8.2 posthog==3.5.0 pre-commit==3.6.2 prompt-toolkit==3.0.43 proto-plus==1.23.0 protobuf==4.25.3 psycopg==3.1.18 psycopg2-binary==2.9.9 ptyprocess==0.7.0 pure-eval==0.2.2 pyasn1==0.5.1 pyasn1-modules==0.3.0 pycparser==2.21 pycryptodome==3.20.0 pydantic==2.6.4 pydantic_core==2.16.3 Pygments==2.17.2 PyJWT==2.8.0 pymongo==4.6.2 pymssql==2.2.11 PyMySQL==1.1.0 PySocks==1.7.1 pytest==7.4.3 pytest-asyncio==0.23.5 pytest-cov==4.1.0 python-arango==7.9.1 python-dateutil==2.9.0.post0 python-keycloak==3.9.1 pytz==2024.1 PyYAML==6.0.1 qdrant-client==1.8.2 reactivex==4.0.4 readme_renderer==43.0 redis==5.0.3 requests==2.31.0 requests-oauthlib==1.4.0 requests-toolbelt==1.0.0 rfc3986==2.0.0 rich==13.7.1 rsa==4.9 s3transfer==0.10.0 scramp==1.4.4 SecretStorage==3.3.3 selenium==4.18.1 six==1.16.0 sniffio==1.3.1 snowballstemmer==2.2.0 sortedcontainers==2.4.0 Sphinx==7.2.6 sphinxcontrib-applehelp==1.0.8 sphinxcontrib-devhelp==1.0.6 sphinxcontrib-htmlhelp==2.0.5 sphinxcontrib-jsmath==1.0.1 sphinxcontrib-qthelp==1.0.7 sphinxcontrib-serializinghtml==1.1.10 SQLAlchemy==2.0.28 stack-data==0.6.3 tenacity==8.2.3 -e @.**@.#egg=testcontainers tomli==2.0.1 traitlets==5.14.3 trio==0.24.0 trio-websocket==0.11.1 twine==4.0.2 typing_extensions==4.11.0 tzlocal==5.2 urllib3==1.26.18 validators==0.22.0 virtualenv==20.25.1 wcwidth==0.2.13 weaviate-client==4.5.4 websocket-client==1.7.0 wrapt==1.16.0 wsproto==1.2.0 zipp==3.17.0 @.**@.#egg=testcontainerstomli==2.0.1traitlets==5.14.3trio==0.24.0trio-websocket==0.11.1twine==4.0.2typing_extensions==4.11.0tzlocal==5.2urllib3==1.26.18validators==0.22.0virtualenv==20.25.1wcwidth==0.2.13weaviate-client==4.5.4websocket-client==1.7.0wrapt==1.16.0wsproto==1.2.0zipp==3.17.0>

  • Docker

Client: Docker Engine - Community Version: 26.1.2 Context: rootless Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc.) Version: v0.14.0 Path: /usr/libexec/docker/cli-plugins/docker-buildx compose: Docker Compose (Docker Inc.) Version: v2.27.0 Path: /usr/libexec/docker/cli-plugins/docker-compose

Server: Containers: 3 Running: 1 Paused: 0 Stopped: 2 Images: 7 Server Version: 25.0.4 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Using metacopy: false Native Overlay Diff: false userxattr: true Logging Driver: json-file Cgroup Driver: systemd Cgroup Version: 2 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 runc Default Runtime: runc Init Binary: docker-init containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb runc version: v1.1.12-0-g51d5e94 init version: de40ad0 Security Options: seccomp Profile: builtin rootless cgroupns Kernel Version: 5.15.0-100-generic Operating System: Ubuntu 22.04.4 LTS OSType: linux Architecture: x86_64 CPUs: 2 Total Memory: 7.752GiB Name: bow ID: 76ee101e-0272-4884-925b-467f3395fa5f Docker Root Dir: /home/roy/.local/share/docker Debug Mode: false Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false

WARNING: No cpu cfs quota support WARNING: No cpu cfs period support WARNING: No cpu shares support WARNING: No cpuset support WARNING: No io.weight support WARNING: No io.weight (per device) support WARNING: No io.max (rbps) support WARNING: No io.max (wbps) support WARNING: No io.max (riops) support WARNING: No io.max (wiops) support

— Reply to this email directly, view it on GitHub https://github.com/testcontainers/testcontainers-python/issues/570, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACECGJA2XON346VRTSYCYA3ZB7B3FAVCNFSM6AAAAABHTBZ6OCVHI2DSMVQWIX3LMV43ASLTON2WKOZSGI4TCNBXG4ZTCOI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

Tranquility2 commented 3 months ago

Understood, PR is up (just removing those lines, if you prefer something else please update me)

alexanderankin commented 3 months ago

Merged! I think the more useful test is with the typical input, and the test should tolerate the warning, but by too small of a margin to matter.

If for some reason it breaks the other way, hopefully we will have gained enough adoption by then that someone opens an issue and we implement that test then.

On Sun, May 12, 2024, 4:00 PM Roy Moore @.***> wrote:

Understood, PR is up (just removing those lines, if you prefer something else please update me)

— Reply to this email directly, view it on GitHub https://github.com/testcontainers/testcontainers-python/issues/570#issuecomment-2106358836, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACECGJEU3Q6DNMGKCTIWAWTZB7C5BAVCNFSM6AAAAABHTBZ6OCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBWGM2TQOBTGY . You are receiving this because you commented.Message ID: @.***>

Tranquility2 commented 3 months ago

I agree, in any case I'll be happy to update the test to something more robust if/when needed. (right now this just blocked me from working on some other stuff 😅)