jneight / django-db-geventpool

Another DB pool using gevent
Apache License 2.0
167 stars 29 forks source link

AttributeError: 'PostgresConnectionPool' object has no attribute 'open' in Django 5.1 #93

Open johnnymetz opened 1 month ago

johnnymetz commented 1 month ago

Background

After upgrading from django-db-geventpool 4.0.6 to 4.0.7, I'm seeing an error when running the Django migrations in Django 5.1.

Steps to Reproduce

Requirements:

$ pip freeze
asgiref==3.8.1
build==1.2.1
click==8.1.7
dj-database-url==2.2.0
Django==5.1
django-db-geventpool==4.0.7
gevent==24.2.1
greenlet==3.0.3
packaging==24.1
pip-tools==7.4.1
psycopg==3.2.1
psycopg-binary==3.2.1
pyproject_hooks==1.1.0
sqlparse==0.5.1
typing_extensions==4.12.2
zope.event==5.0
zope.interface==7.0.1

Database settings:

import dj_database_url

DATABASE_URL = "postgres://postgres:postgres@localhost/postgres"
DATABASES = {
    "default": {
        **dj_database_url.parse(DATABASE_URL),
        "ENGINE": "django_db_geventpool.backends.postgresql_psycopg3",
    }
}

Run migrations:

$ python manage.py migrate
Traceback (most recent call last):
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/./manage.py", line 22, in <module>
    main()
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/./manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/core/management/base.py", line 413, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/core/management/base.py", line 459, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/core/management/base.py", line 107, in wrapper
    res = handle_func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/core/management/commands/migrate.py", line 118, in handle
    executor = MigrationExecutor(connection, self.migration_progress_callback)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/db/migrations/executor.py", line 18, in __init__
    self.loader = MigrationLoader(self.connection)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/db/migrations/loader.py", line 58, in __init__
    self.build_graph()
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/db/migrations/loader.py", line 235, in build_graph
    self.applied_migrations = recorder.applied_migrations()
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/db/migrations/recorder.py", line 89, in applied_migrations
    if self.has_table():
       ^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/db/migrations/recorder.py", line 63, in has_table
    with self.connection.cursor() as cursor:
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/db/backends/base/base.py", line 320, in cursor
    return self._cursor()
           ^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/db/backends/base/base.py", line 296, in _cursor
    self.ensure_connection()
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/db/backends/base/base.py", line 279, in ensure_connection
    self.connect()
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/db/backends/base/base.py", line 256, in connect
    self.connection = self.get_new_connection(conn_params)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django_db_geventpool/backends/base.py", line 41, in get_new_connection
    self.connection = self.pool.get()
                      ^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django_db_geventpool/backends/pool.py", line 55, in get
    conn = self.create_connection()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django_db_geventpool/backends/postgresql_psycopg3/base.py", line 29, in create_connection
    conn = self.connect(*self.args, **self.kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django_db_geventpool/backends/base.py", line 33, in <lambda>
    self._pool = self.pool_class(super(), **self.get_connection_params(), connect=lambda parent, **kw: parent.get_new_connection(conn_params=kw))
                                                                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/johnnymetz/Documents/stuffs/py-stuff/mysite/venv/lib/python3.11/site-packages/django/db/backends/postgresql/base.py", line 347, in get_new_connection
    self.pool.open()
    ^^^^^^^^^^^^^^
AttributeError: 'PostgresConnectionPool' object has no attribute 'open'

Notes

jneight commented 1 month ago

It seems that in django 5.1, when using psycopg3, there is now support for using psycopg ConnectionPool directly, see this doc: https://docs.djangoproject.com/en/5.1/ref/databases/#connection-pool

Can you try setting pool as False?

"OPTIONS": {
    "pool": False,
},
johnnymetz commented 3 weeks ago

That didn't fix it unfortunately

yokotoka commented 3 weeks ago

Same error, on 4.0.6 all great but broken on 4.0.7

MaxwellDPS commented 3 weeks ago

Confirming seeing the same issue on Django 5 and 4.0.7