search

testcontainers / testcontainers-python

Testcontainers is a Python library that providing a friendly API to run Docker container. It is designed to create runtime environment to use during your automatic tests.
https://testcontainers-python.readthedocs.io/en/latest/
Apache License 2.0
1.61k stars 296 forks source link

Postgres example fails with OperationalError #253

Closed chbndrhnns closed 6 months ago

chbndrhnns commented 2 years ago

When copy and pasting the example for postgres, I get this output:

/Users/jo/Library/Caches/pypoetry/virtualenvs/tc-mVmN3aPa-py3.10/bin/python /Users/jo/src/scratch/testcontainers/tc/main.py 
Pulling image postgres:9.5
Container started: 7044b0ef12a2
Waiting to be ready...
Waiting to be ready...
Waiting to be ready...
Waiting to be ready...
Waiting to be ready...
Waiting to be ready...
Waiting to be ready...
PostgreSQL 9.5.25 on aarch64-unknown-linux-gnu (Debian 9.5.25-1.pgdg90+1), compiled by gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516, 64-bit
Exception during reset or similar
Traceback (most recent call last):
  File "/Users/jo/Library/Caches/pypoetry/virtualenvs/tc-mVmN3aPa-py3.10/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 753, in _finalize_fairy
    fairy._reset(pool)
  File "/Users/jo/Library/Caches/pypoetry/virtualenvs/tc-mVmN3aPa-py3.10/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 1004, in _reset
    pool._dialect.do_rollback(self)
  File "/Users/jo/Library/Caches/pypoetry/virtualenvs/tc-mVmN3aPa-py3.10/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 683, in do_rollback
    dbapi_connection.rollback()
psycopg2.OperationalError: server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.

Process finished with exit code 0

Is there anything else that needs to be configured?

tillahoffmann commented 2 years ago

Are you referring to the example here? A few questions regarding your environment:

chbndrhnns commented 2 years ago

Yes, I copy & pasted the snippet you linked.

  • What's your operating system and architecture (it looks like a Mac from your paths; is this Apple Silicon or Intel)? Apple M1, macOS 12.6
  • What's your python version? 3.9.12
  • What's your testcontainers-python version? 3.7.0
tillahoffmann commented 2 years ago

Great, thanks. And what's the output of running the following from the command line (see below for my result)?

$ docker pull postgres:9.5
9.5: Pulling from library/postgres
Digest: sha256:75ebf479151a8fd77bf2fed46ef76ce8d518c23264734c48f2d1de42b4eb40ae
Status: Image is up to date for postgres:9.5
docker.io/library/postgres:9.5
chbndrhnns commented 2 years ago

It's very similar for me:

❯ docker pull postgres:9.5
9.5: Pulling from library/postgres
Digest: sha256:75ebf479151a8fd77bf2fed46ef76ce8d518c23264734c48f2d1de42b4eb40ae
Status: Image is up to date for postgres:9.5
docker.io/library/postgres:9.5
idkosilov commented 2 years ago

Same issue. Are there any updates?

pffijt commented 2 years ago

I am not having this issue. It seems that sqlalchemy brings up this exception when the script tries to close the container. Maybe it has to do that the Postgresql container gets removed with force??

tillahoffmann commented 1 year ago

@idkosilov, @chbndrhnns, would either of you be up for sending a PR with a test that illustrates the issue?

chbndrhnns commented 1 year ago

@tillahoffmann Here is a repository which has the issue. Is that what you were looking for?

tillahoffmann commented 1 year ago

Hm, the code in the repo you shared is running fine on my machine. Could you provide the information in the updated bug report template here?

chbndrhnns commented 1 year ago 
# Get the operating system information (on a unix os).
uname -a
Darwin Jos-MacBook-Pro.local 21.6.0 Darwin Kernel Version 21.6.0: Mon Aug 22 20:19:52 PDT 2022; root:xnu-8020.140.49~2/RELEASE_ARM64_T6000 arm64

# Get the python version.
$ python --version
Python 3.11.0

# Get the docker version and other docker information.
$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.9.1)
  compose: Docker Compose (Docker Inc., v2.13.0)
  dev: Docker Dev Environments (Docker Inc., v0.0.5)
  extension: Manages Docker extensions (Docker Inc., v0.2.16)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.22.0)

Server:
 Containers: 10
  Running: 0
  Paused: 0
  Stopped: 10
 Images: 25
 Server Version: 20.10.21
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 770bd0108c32f3fb5c73ae1264f7e503fe7b2661
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.15.49-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 5
 Total Memory: 7.765GiB
 Name: docker-desktop
 ID: <snip>
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Live Restore Enabled: false

# Get all python packages.
$ pip freeze
-e git+ssh://git@github.com/chbndrhnns/aoc-py.git@c98e3f8bbe42f001e4956af262bfa243088bf0d2#egg=aoc22_py
attrs==22.1.0
black==22.8.0
certifi==2022.9.24
charset-normalizer==2.1.1
click==8.1.3
deprecation==2.1.0
docker==6.0.0
idna==3.4
libcst==0.4.7
moreorless==0.4.0
mypy-extensions==0.4.3
packaging==21.3
pathspec==0.10.1
platformdirs==2.5.2
psycopg2-binary==2.9.3
pyparsing==3.0.9
PyYAML==6.0
requests==2.28.1
SQLAlchemy==1.4.41
stdlibs==2022.6.8
-e git+ssh://git@github.com/chbndrhnns/testcontainers-235.git@d8418545c3aee7f0d91c04908585e0aecb547ff7#egg=tc
testcontainers==3.7.0
toml==0.10.2
tomlkit==0.11.5
trailrunner==1.2.1
typing-inspect==0.8.0
typing_extensions==4.3.0
ufmt==2.0.1
urllib3==1.26.12
usort==1.0.5
websocket-client==1.4.1
wrapt==1.14.1
tillahoffmann commented 1 year ago

Would you mind giving it another go after upgrading to the most recent testcontainers version 3.7.1 (or 4.0.0rc1)?

chbndrhnns commented 1 year ago

I tried both 3.7.1 and this command (didn't know how exactly I could install 4.0.0rc1). The result is still the same:

poetry add git+ssh://git@github.com/testcontainers/testcontainers-python@master#subdirectory=postgres

jweckman commented 1 year ago

Same problem on Arch Linux.

I used the same repo and tried installing both using testcontainers[postgres] method and testcontainers-postgres using latest version picked up by poetry. Same result in both cases. I have other more complex projects running python and postgres containers just fine on the same system.


# Get the operating system information (on a unix os).
uname -a
Linux host 6.1.7-arch1-1 #1 SMP PREEMPT_DYNAMIC Wed, 18 Jan 2023 19:54:38 +0000 x86_64 GNU/Linux

# Get the python version.
$ python --version
Python 3.10.9

# Get the docker version and other docker information.
$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  compose: Docker Compose (Docker Inc., 2.14.2)

Server:
 Containers: 41
  Running: 0
  Paused: 0
  Stopped: 41
 Images: 104
 Server Version: 20.10.22
 Storage Driver: btrfs
  Build Version: Btrfs v6.0.2
  Library Version: 102
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 9ba4b250366a5ddde94bb7c9d1def331423aa323.m
 runc version:
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 6.1.7-arch1-1
 Operating System: Arch Linux
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 15.53GiB
 ID: DSZP:QKTY:ZI3X:N624:HFGD:DYXL:WNR2:MUGZ:B66D:MJTS:NTBN:QJ52
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

# Get all python packages.
attrs                   22.1.0    Classes Without Boilerplate
black                   22.8.0    The uncompromising code formatter.
certifi                 2022.9.24 Python package for providing Mozilla's CA Bundle.
charset-normalizer      2.1.1     The Real First Universal Charset Detector. Open, modern and actively maintained alternative to Chardet.
click                   8.1.3     Composable command line interface toolkit
docker                  6.0.0     A Python library for the Docker Engine API.
greenlet                1.1.3     Lightweight in-process concurrent programming
idna                    3.4       Internationalized Domain Names in Applications (IDNA)
libcst                  0.4.7     A concrete syntax tree with AST-like properties for Python 3.5, 3.6, 3.7, 3.8, 3.9, and 3.10 programs.
moreorless              0.4.0     Python diff wrapper
mypy-extensions         0.4.3     Experimental type system extensions for programs checked with the mypy typechecker.
packaging               21.3      Core utilities for Python packages
pathspec                0.10.1    Utility library for gitignore style pattern matching of file paths.
platformdirs            2.5.2     A small Python module for determining appropriate platform-specific dirs, e.g. a "user data dir".
psycopg2-binary         2.9.3     psycopg2 - Python-PostgreSQL Database Adapter
pyparsing               3.0.9     pyparsing module - Classes and methods to define and execute parsing grammars
PyYAML                  6.0       YAML parser and emitter for Python
requests                2.28.1    Python HTTP for Humans.
SQLAlchemy              1.4.41    Database Abstraction Library
stdlibs                 2022.6.8  List of packages in the stdlib
testcontainers-core     0.0.1rc1  Core component of testcontainers-python.
testcontainers-postgres 0.0.1rc1  PostgreSQL component of testcontainers-python.
toml                    0.10.2    Python Library for Tom's Obvious, Minimal Language
tomli                   2.0.1     A lil' TOML parser
tomlkit                 0.11.5    Style preserving TOML library
trailrunner             1.2.1     Run things on paths
typing-extensions       4.3.0     Backported and Experimental Type Hints for Python 3.7+
typing-inspect          0.8.0     Runtime inspection utilities for typing module.
ufmt                    2.0.1     Safe, atomic formatting with black and µsort
urllib3                 1.26.12   HTTP library with thread-safe connection pooling, file post, and more.
usort                   1.0.5     A small, safe import sorter
websocket-client        1.4.1     WebSocket client for Python with low level API options
wrapt                   1.14.1    Module for decorators, wrappers and monkey patching.
joarobles commented 1 year ago

Same issue here using testcontainers-compose 0.0.1rc1 on Apple M1:

compose = DockerCompose(
  "tests/compose",
  compose_file_name=["postgres.yaml"],  # type: ignore
  pull=True,
)
version: '3.8'

services:

  test_postgres:
    image: postgres:14
    environment:
      POSTGRES_PASSWORD: root
      TZ: UTC
      PGTZ: UTC
    ports:
      - 5432
    volumes:
      - $LOCAL_WORKSPACE_FOLDER/.devcontainer/postgresql/dumps:/docker-entrypoint-initdb.d
jborman-stonex commented 6 months ago

I've encountered this issue a few times in the past before discovering testcontainers and recently ran into it again while using this package. Try adding a time.sleep() call before attempting to connect to the container, see my example fixture below.

@pytest.fixture(name="db_container", scope="session")
def db_container_fixture() -> Generator[PostgresContainer, None, None]:
    pg_container = PostgresContainer(image="postgres:16.2", dbname="postgres")

    import time; time.sleep(0.0)  # briefly pause to allow the container to restart before connecting
    with pg_container:
        yield pg_container

The postgres server on startup will perform a full reboot cycle prior to accepting incoming connections. If you attempt to connect before it completes the cycle any connection requests will be rejected. I believe this race condition is why the issue is intermittent and waiting a moment before attempting a connect generally resolves the issue.

alexanderankin commented 6 months ago

@jborman-stonex This is a two year old issue, please edit your comment with details about your environment or open a new issue.

alexanderankin commented 6 months ago

For example make sure you are using testcontainers[postgres] and not testcontainers-postgres or whatever

chbndrhnns commented 6 months ago

For me, the issue is solved using this setup on an M1 Pro macbook.

import sqlalchemy
from sqlalchemy import text

from testcontainers.postgres import PostgresContainer

if __name__ == "__main__":
    postgres_container = PostgresContainer("postgres:9.5")
    with postgres_container as postgres:
        engine = sqlalchemy.create_engine(postgres.get_connection_url())
        with engine.connect() as connection:
            stmt = text("select version()")
            result = connection.execute(stmt)
            (version,) = result.fetchone()
        print(version)
> pip freeze
[...]
psycopg2==2.9.9
SQLAlchemy==2.0.30
testcontainers==4.4.0