search

testcontainers / testcontainers-python

Testcontainers is a Python library that providing a friendly API to run Docker container. It is designed to create runtime environment to use during your automatic tests.
https://testcontainers-python.readthedocs.io/en/latest/
Apache License 2.0
1.61k stars 296 forks source link

Bug: Testcontainer with ryuk version 0.10.0 not working in Apple Mac M1 Chip #744

Open debu999 opened 4 days ago

debu999 commented 4 days ago

Describe the bug testcontainer does not start with version 0.10.0 ryuk in apple mac m1 chip.

Refer to the reproducer at tc-python-version-bug branch: main default 0.8.1 ryuk working branch: feature/tc-ryuk-0.9.0 ryuk working branch: feature/tc-ryuk-0.10.0 ryuk not working

To Reproduce

Refer to the reproducer at tc-python-version-bug branch: main default 0.8.1 ryuk working branch: feature/tc-ryuk-0.9.0 ryuk working branch: feature/tc-ryuk-0.10.0 ryuk not working

/Volumes/../projects/tc-python-version-bug/.venv/bin/python /Users/../Library/Application Support/JetBrains/IntelliJIdea2024.3/plugins/python-ce/helpers/pycharm/_jb_pytest_runner.py --path /Volumes/../projects/tc-python-version-bug/tests/test_messaging.py 
Testing started at 10:50 pm ...
Launching pytest with arguments /Volumes/../projects/tc-python-version-bug/tests/test_messaging.py --no-header --no-summary -q in /Volumes/../projects/tc-python-version-bug/tests

============================= test session starts ==============================
collecting ... collected 2 items

test_messaging.py::test_produce_message ERROR                            [ 50%]Pulling image testcontainers/ryuk:0.10.0
Container started: f936853656f4

test setup failed
self = <docker.api.client.APIClient object at 0x108931310>
response = <Response [404]>

    def _raise_for_status(self, response):
        """Raises stored :class:`APIError`, if one occurred."""
        try:
>           response.raise_for_status()

../.venv/lib/python3.13/site-packages/docker/api/client.py:275: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Response [404]>

    def raise_for_status(self):
        """Raises :class:`HTTPError`, if one occurred."""

        http_error_msg = ""
        if isinstance(self.reason, bytes):
            # We attempt to decode utf-8 first because some servers
            # choose to localize their reason strings. If the string
            # isn't utf-8, we fall back to iso-8859-1 for all other
            # encodings. (See PR #3538)
            try:
                reason = self.reason.decode("utf-8")
            except UnicodeDecodeError:
                reason = self.reason.decode("iso-8859-1")
        else:
            reason = self.reason

        if 400 <= self.status_code < 500:
            http_error_msg = (
                f"{self.status_code} Client Error: {reason} for url: {self.url}"
            )

        elif 500 <= self.status_code < 600:
            http_error_msg = (
                f"{self.status_code} Server Error: {reason} for url: {self.url}"
            )

        if http_error_msg:
>           raise HTTPError(http_error_msg, response=self)
E           requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.47/containers/f936853656f43cb5bbd291794a349e5fd76665a52b51dc1c11979a253c6f3f22/json

../.venv/lib/python3.13/site-packages/requests/models.py:1024: HTTPError

The above exception was the direct cause of the following exception:

    @pytest.fixture(scope="session", name="kafka_broker")
    def kafka_container():
      """
      Test suite for the messaging producer functionality using pytest and Redpanda as the Kafka broker.
      This module includes fixtures and test cases to validate message production and delivery reports.

      Fixtures:
          kafka_container: A pytest fixture that sets up a Redpanda container for testing.

      Tests:
          test_multiply: A simple test to validate multiplication logic.
          test_produce_message: Tests the message production functionality
          by producing a message to the Kafka topic.

      Examples:
          To run the tests, use the pytest command in the terminal.
      """

>     with RedpandaContainer(
          image="docker.redpanda.com/redpandadata/redpanda:v24.2.11") as redpanda:

test_messaging.py:55: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../.venv/lib/python3.13/site-packages/testcontainers/core/container.py:126: in __enter__
    return self.start()
../.venv/lib/python3.13/site-packages/testcontainers/kafka/_redpanda.py:70: in start
    super().start()
../.venv/lib/python3.13/site-packages/testcontainers/core/container.py:89: in start
    Reaper.get_instance()
../.venv/lib/python3.13/site-packages/testcontainers/core/container.py:207: in get_instance
    Reaper._instance = Reaper._create_instance()
../.venv/lib/python3.13/site-packages/testcontainers/core/container.py:238: in _create_instance
    wait_for_logs(Reaper._container, r".* Started!")
../.venv/lib/python3.13/site-packages/testcontainers/core/waiting_utils.py:109: in wait_for_logs
    stdout = container.get_logs()[0].decode()
../.venv/lib/python3.13/site-packages/testcontainers/core/container.py:187: in get_logs
    return self._container.logs(stderr=False), self._container.logs(stdout=False)
../.venv/lib/python3.13/site-packages/docker/models/containers.py:322: in logs
    return self.client.api.logs(self.id, **kwargs)
../.venv/lib/python3.13/site-packages/docker/utils/decorators.py:19: in wrapped
    return f(self, resource_id, *args, **kwargs)
../.venv/lib/python3.13/site-packages/docker/api/container.py:896: in logs
    output = self._get_result(container, stream, res)
../.venv/lib/python3.13/site-packages/docker/api/client.py:483: in _get_result
    return self._get_result_tty(stream, res, self._check_is_tty(container))
../.venv/lib/python3.13/site-packages/docker/utils/decorators.py:19: in wrapped
    return f(self, resource_id, *args, **kwargs)
../.venv/lib/python3.13/site-packages/docker/api/client.py:479: in _check_is_tty
    cont = self.inspect_container(container)
../.venv/lib/python3.13/site-packages/docker/utils/decorators.py:19: in wrapped
    return f(self, resource_id, *args, **kwargs)
../.venv/lib/python3.13/site-packages/docker/api/container.py:793: in inspect_container
    return self._result(
../.venv/lib/python3.13/site-packages/docker/api/client.py:281: in _result
    self._raise_for_status(response)
../.venv/lib/python3.13/site-packages/docker/api/client.py:277: in _raise_for_status
    raise create_api_error_from_http_exception(e) from e
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

e = HTTPError('404 Client Error: Not Found for url: http+docker://localhost/v1.47/containers/f936853656f43cb5bbd291794a349e5fd76665a52b51dc1c11979a253c6f3f22/json')

    def create_api_error_from_http_exception(e):
        """
        Create a suitable APIError from requests.exceptions.HTTPError.
        """
        response = e.response
        try:
            explanation = response.json()['message']
        except ValueError:
            explanation = (response.text or '').strip()
        cls = APIError
        if response.status_code == 404:
            explanation_msg = (explanation or '').lower()
            if any(fragment in explanation_msg
                   for fragment in _image_not_found_explanation_fragments):
                cls = ImageNotFound
            else:
                cls = NotFound
>       raise cls(e, response=response, explanation=explanation) from e
E       docker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.47/containers/f936853656f43cb5bbd291794a349e5fd76665a52b51dc1c11979a253c6f3f22/json: Not Found ("No such container: f936853656f43cb5bbd291794a349e5fd76665a52b51dc1c11979a253c6f3f22")

../.venv/lib/python3.13/site-packages/docker/errors.py:39: NotFound

test_messaging.py::test_consume_message ERROR                            [100%]
test setup failed
self = <docker.api.client.APIClient object at 0x108931310>
response = <Response [404]>

    def _raise_for_status(self, response):
        """Raises stored :class:`APIError`, if one occurred."""
        try:
>           response.raise_for_status()

../.venv/lib/python3.13/site-packages/docker/api/client.py:275: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Response [404]>

    def raise_for_status(self):
        """Raises :class:`HTTPError`, if one occurred."""

        http_error_msg = ""
        if isinstance(self.reason, bytes):
            # We attempt to decode utf-8 first because some servers
            # choose to localize their reason strings. If the string
            # isn't utf-8, we fall back to iso-8859-1 for all other
            # encodings. (See PR #3538)
            try:
                reason = self.reason.decode("utf-8")
            except UnicodeDecodeError:
                reason = self.reason.decode("iso-8859-1")
        else:
            reason = self.reason

        if 400 <= self.status_code < 500:
            http_error_msg = (
                f"{self.status_code} Client Error: {reason} for url: {self.url}"
            )

        elif 500 <= self.status_code < 600:
            http_error_msg = (
                f"{self.status_code} Server Error: {reason} for url: {self.url}"
            )

        if http_error_msg:
>           raise HTTPError(http_error_msg, response=self)
E           requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.47/containers/f936853656f43cb5bbd291794a349e5fd76665a52b51dc1c11979a253c6f3f22/json

../.venv/lib/python3.13/site-packages/requests/models.py:1024: HTTPError

The above exception was the direct cause of the following exception:

    @pytest.fixture(scope="session", name="kafka_broker")
    def kafka_container():
      """
      Test suite for the messaging producer functionality using pytest and Redpanda as the Kafka broker.
      This module includes fixtures and test cases to validate message production and delivery reports.

      Fixtures:
          kafka_container: A pytest fixture that sets up a Redpanda container for testing.

      Tests:
          test_multiply: A simple test to validate multiplication logic.
          test_produce_message: Tests the message production functionality
          by producing a message to the Kafka topic.

      Examples:
          To run the tests, use the pytest command in the terminal.
      """

>     with RedpandaContainer(
          image="docker.redpanda.com/redpandadata/redpanda:v24.2.11") as redpanda:

test_messaging.py:55: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../.venv/lib/python3.13/site-packages/testcontainers/core/container.py:126: in __enter__
    return self.start()
../.venv/lib/python3.13/site-packages/testcontainers/kafka/_redpanda.py:70: in start
    super().start()
../.venv/lib/python3.13/site-packages/testcontainers/core/container.py:89: in start
    Reaper.get_instance()
../.venv/lib/python3.13/site-packages/testcontainers/core/container.py:207: in get_instance
    Reaper._instance = Reaper._create_instance()
../.venv/lib/python3.13/site-packages/testcontainers/core/container.py:238: in _create_instance
    wait_for_logs(Reaper._container, r".* Started!")
../.venv/lib/python3.13/site-packages/testcontainers/core/waiting_utils.py:109: in wait_for_logs
    stdout = container.get_logs()[0].decode()
../.venv/lib/python3.13/site-packages/testcontainers/core/container.py:187: in get_logs
    return self._container.logs(stderr=False), self._container.logs(stdout=False)
../.venv/lib/python3.13/site-packages/docker/models/containers.py:322: in logs
    return self.client.api.logs(self.id, **kwargs)
../.venv/lib/python3.13/site-packages/docker/utils/decorators.py:19: in wrapped
    return f(self, resource_id, *args, **kwargs)
../.venv/lib/python3.13/site-packages/docker/api/container.py:896: in logs
    output = self._get_result(container, stream, res)
../.venv/lib/python3.13/site-packages/docker/api/client.py:483: in _get_result
    return self._get_result_tty(stream, res, self._check_is_tty(container))
../.venv/lib/python3.13/site-packages/docker/utils/decorators.py:19: in wrapped
    return f(self, resource_id, *args, **kwargs)
../.venv/lib/python3.13/site-packages/docker/api/client.py:479: in _check_is_tty
    cont = self.inspect_container(container)
../.venv/lib/python3.13/site-packages/docker/utils/decorators.py:19: in wrapped
    return f(self, resource_id, *args, **kwargs)
../.venv/lib/python3.13/site-packages/docker/api/container.py:793: in inspect_container
    return self._result(
../.venv/lib/python3.13/site-packages/docker/api/client.py:281: in _result
    self._raise_for_status(response)
../.venv/lib/python3.13/site-packages/docker/api/client.py:277: in _raise_for_status
    raise create_api_error_from_http_exception(e) from e
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

e = HTTPError('404 Client Error: Not Found for url: http+docker://localhost/v1.47/containers/f936853656f43cb5bbd291794a349e5fd76665a52b51dc1c11979a253c6f3f22/json')

    def create_api_error_from_http_exception(e):
        """
        Create a suitable APIError from requests.exceptions.HTTPError.
        """
        response = e.response
        try:
            explanation = response.json()['message']
        except ValueError:
            explanation = (response.text or '').strip()
        cls = APIError
        if response.status_code == 404:
            explanation_msg = (explanation or '').lower()
            if any(fragment in explanation_msg
                   for fragment in _image_not_found_explanation_fragments):
                cls = ImageNotFound
            else:
                cls = NotFound
>       raise cls(e, response=response, explanation=explanation) from e
E       docker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.47/containers/f936853656f43cb5bbd291794a349e5fd76665a52b51dc1c11979a253c6f3f22/json: Not Found ("No such container: f936853656f43cb5bbd291794a349e5fd76665a52b51dc1c11979a253c6f3f22")

../.venv/lib/python3.13/site-packages/docker/errors.py:39: NotFound

========================= 2 errors in 71.06s (0:01:11) =========================

Process finished with exit code 1

Runtime environment

Provide a summary of your runtime environment. MacOS python 3.13 docker Docker version 27.3.1, build ce12230 testcontainers = "^4.8.2" Which operating system, python version, and docker version are you using? What is the version of testcontainers-python you are using? You can run the following commands to get the relevant information.

# Get the operating system information (on a unix os).
$ uname -a
uname -a
Darwin Mac-mini.local 24.0.0 Darwin Kernel Version 24.0.0: Tue Sep 24 23:36:26 PDT 2024; root:xnu-11215.1.12~1/RELEASE_ARM64_T8103 arm64
# Get the python version.
$ python --version
Python 3.13.0
# Get the docker version and other docker information.
$ docker info
docker info
Client:
 Version:    27.3.1
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.17.1-desktop.1
    Path:     /Users/../.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.7-desktop.1
    Path:     /Users/../.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.37
    Path:     /Users/../.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Alpha) (Docker Inc.)
    Version:  v0.0.15
    Path:     /Users/../.docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/../.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.27
    Path:     /Users/../.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     /Users/../.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.3.0
    Path:     /Users/../.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/../.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.14.0
    Path:     /Users/../.docker/cli-plugins/docker-scout
WARNING: Plugin "/Users/../.docker/cli-plugins/docker-scan" is not valid: failed to fetch metadata: fork/exec /Users/../.docker/cli-plugins/docker-scan: no such file or directory

Server:
 Containers: 36
  Running: 22
  Paused: 0
  Stopped: 14
 Images: 26
 Server Version: 27.3.1
 Storage Driver: stargz
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 472731909fa34bd7bc9c087e4c27943f9835f111
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.10.11-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 8
 Total Memory: 5.786GiB
 Name: docker-desktop
 ID: 9c45f5c5-734b-4b90-b6cc-2a07e65eb446
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/../Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: true
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile
# Get all python packages.
$ pip freeze
pip freeze                                                                                           tc-python-version-bug-py3.13 
certifi==2024.8.30
charset-normalizer==3.4.0
confluent-kafka==2.6.1
coverage==7.6.8
docker==7.1.0
idna==3.10
iniconfig==2.0.0
packaging==24.2
pluggy==1.5.0
pytest==8.3.3
pytest-asyncio==0.24.0
requests==2.32.3
-e git+ssh://git@github.com/../tc-python-version-bug.git@ac4e2449da8034a2cb412946352974442cdda128#egg=tc_python_version_bug
testcontainers==4.8.2
typing_extensions==4.12.2
urllib3==2.2.3
wrapt==1.17.0
debu999 commented 4 days ago

As per the comment in https://github.com/testcontainers/moby-ryuk/issues/177 I have attached the reproducer to help figure the issue in the program. Please advise on it.