different behaviour compared to local docker usage

[philipphanemann]

Hello, I want to test a postgres database, which runs in a docker container. Unfortunately, it works with a local docker instance running but not when I use the plugins pytest-docker and pytest-docker-compose. To present a minimal example I have two files.

1. The docker-compose.yml file, which contains the following:

version: '2'
services:
  postgres:
    image: "postgres"
    ports:
      - "54320:5432"
    environment:
      - POSTGRES_PASSWORD=mysecretpass
      - POSTGRES_DB=immo
    container_name: test_postgres

2. The necessary fixtures and the test are put in one file called test_mini.py, which is in the same directory as the docker-compose file . For convenience, I use the library sqlachemy_utils, which can be simply installed with pip. Additionally, the 'mainstream' database driver "psycopg2" is needed, which can also be installed with pip. The file contains:

import os
import pytest
from sqlalchemy_utils import database_exists

@pytest.fixture(scope="session")
def docker_compose_file(pytestconfig):
    return os.path.join(".", "docker-compose.yml")

@pytest.fixture(scope="session")
def db_url(docker_ip, docker_services):
    """ default database for testing """

    port = docker_services.port_for("postgres", 5432)
    return f'postgresql://postgres:mysecretpass@{docker_ip}:{port}/immo'

def test_db(db_url):
    assert database_exists(db_url)

The resulting error is:

E       sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) server closed the connection unexpectedly
E           This probably means the server terminated abnormally
E           before or while processing the request.
E       
E       (Background on this error at: http://sqlalche.me/e/e3q8)

As stated above, this only becomes an error, if I use the pytest-docker plugin. Furthermore, if I put a debugging statement before the assert import pdb; pdb.set_trace() and simply continue c, the test suddenly succeeds. I have looked through the web, but couldn't find a solution. Do you have an idea? I would appreciate that very much.

Best

[AndreLouisCaron]

This is likely caused by a timing issue: the Docker container is spawned before your test_db() function is invoked, but it has not (yet) finished initializing and the connection fails. You don't encounter this with manual docker compose up because you do the steps more slowly than the automatic test suite.

The solution I use in my test suites that use pytest-docker is to have some kind of polling until the database is responsive. Instead of that assert database_exists(), you should make sure this condition becomes true within some maximum timeout.

[AndreLouisCaron]

Note that the exact error you encounter depends on timing and the operating system. A "connection refused" error is super common.

The "server closed the connection unexpectedly" is frequent when using Docker Desktop on Windows because it runs a TCP proxy to forward traffic to a port on the guest OS. This proxy is usually "immediately" ready and accepts the connection, but finds the Docker container is not (yet) listening on this port and closes the connection.

[philipphanemann]

Thanks a lot for the comment. A simple sleep(1) helped to circumvent the problem. I'll try to find a more elegant way to tackle the "race condition".