Bug: Testcontainers can't connect to set DOCKER_HOST in Gitlab CI

[spreeni]

Describe the bug

I can't get testcontainers to run in my Gitlab CI/CD pipeline, despite following the docker:dind setup, setting DOCKER_HOST="tcp://docker:2375" and DOCKER_TLS_CERTDIR="" (see https://java.testcontainers.org/supported_docker_environment/continuous_integration/gitlab_ci/ or https://samanta-reinosoa.medium.com/testing-fastapi-with-testcontainers-in-gitlab-b7c62068aeef).

testcontainers seems to be unable to connect to the specified docker host, I am getting the error docker.errors.DockerException: Error while fetching server API version: HTTPConnectionPool(host='docker', port=2375): Max retries exceeded with url: /version (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f102d2cd8d0>: Failed to resolve 'docker' ([Errno -2] Name or service not known)"))

To Reproduce Below my .gitlab-ci.yml:

stages:
  - qa

default:
  image: python:3.11

variables:
  PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
  DOCKER_DRIVER: overlay2
  GIT_SSL_NO_VERIFY: "true"
  RYE_VERSION: "0.34.0"

cache:
  paths:
    - .venv/

.before_script: &before_script
    - curl -sSf https://rye.astral.sh/get | RYE_INSTALL_OPTION="--yes" bash
    - . "$HOME/.rye/env"
    - rye sync --no-lock
    - . .venv/bin/activate
    - python -V  # Print out python version for debugging

tests:
  stage: qa
  services:
    - name: docker:dind
      command: [ "--tls=false" ]
  variables:
    DOCKER_TLS_CERTDIR: ""
    DOCKER_HOST: "tcp://docker:2375"
  before_script: *before_script
  script:
    - pytest --cov --cov-report term --cov-report xml:coverage.xml
  coverage: '/TOTAL.*? (100(?:\.0+)?\%\|[1-9]?\d(?:\.\d+)?\%)$/'
  artifacts:
    reports:
      coverage_report:
        coverage_format: cobertura
        path: coverage.xml

Runtime environment

Provide a summary of your runtime environment. Which operating system, python version, and docker version are you using? What is the version of testcontainers-python you are using? You can run the following commands to get the relevant information.

I am using testcontainers-core==0.0.1rc1.

[alexanderankin]

several things to note about versions:

1. dind doesn't work in testcontainers-python v4 - see the pinned issue #517
2. testcontainers-core is an orphaned package - its essentially 4.0.0-rc* (as are testcontainers-mysql etc, use testcontainers[mysql] instead) - we publish to testcontainers regularly (although I am having less and less time to publish i can still accept PRs and release those in that package. - https://pypi.org/project/testcontainers/
3. if no bugs fixed since 3.7.1 affect your setup and you are not using community modules, this may be the best bet for dind.

[spreeni]

Thanks for the super-fast answer! Those are some good infos, thanks for providing them. I tried it with 3.7.1, but had the same issue. I have a gut feeling that this is something in the Gitlab runner set up by our sysadmins.

For now I will test locally with testcontainers, but maybe I will have to move to running Opensearch as a general service for the CI/CD pipeline then.

Hope Gitlab CI/CD support will come in the future - would love to use it in the pipeline.

[alexanderankin]

if you absolutely need it then you just override the function to get the port to detect if you are inside the container and use the port instead of the forwarded port (and couple other gotchas if you are using custom networks, etc) - but in general yes this is not supported yet. there are some people using it but the setups are apparently different. I don't have access to an environment where i'd need something like that.

[gaby]

@spreeni It's not your runner. Testcontainers changed how they detect docker and it no longer works since 4.x.x