search

Bogdanp / django_dramatiq

A Django app that integrates with Dramatiq.
https://dramatiq.io
Other
331 stars 77 forks source link

DbConnectionsMiddleware - FATAL: sorry, too many clients already #76

Closed BradleyKirton closed 3 years ago

BradleyKirton commented 3 years ago

Hi there, first up thanks for creating dramatiq it's really a pleasure to work with.

I have a fairly new project which I have upgraded to Django 3.1.1 and I noticed that the database connections are not being disposed of after the tasks have finished. After some time I realised that if I move the DbConnectionsMiddleware above the AdminMiddleware then connections are correctly disposed of.

The middleware stack that is currently working for me looks like this:

"MIDDLEWARE": [
  "dramatiq.middleware.AgeLimit",
  "dramatiq.middleware.TimeLimit",
  "dramatiq.middleware.Retries",
  "django_dramatiq.middleware.AdminMiddleware",
  "django_dramatiq.middleware.DbConnectionsMiddleware",
]

Previously I had it like this:

"MIDDLEWARE": [
  "dramatiq.middleware.AgeLimit",
  "dramatiq.middleware.TimeLimit",
  "dramatiq.middleware.Retries",
  "django_dramatiq.middleware.DbConnectionsMiddleware",
  "django_dramatiq.middleware.AdminMiddleware",
]

Does this ordering matter? I would assume not since the default and documentation shows it like the latter.

I am not sure what the issue here could be as I have only noticed this now but as I mentioned I have recently upgraded django.

It is working fine for me now that I have switched the ordering but I thought I would bring this to your attention anyway.

Bogdanp commented 3 years ago

The order you now have it in makes more sense. Otherwise, the conn middleware might try to close the connections after a message, but then the admin middleware might get new ones. I'll fix the README.

m000 commented 1 year ago

@BradleyKirton @Bogdanp, I am confused.

@BradleyKirton you mention that if moving DbConnectionsMiddleware above the AdminMiddleware makes the connection disposal work. Then you list your "currently working" middleware block that shows DbConnectionsMiddleware below AdminMiddleware . Which of the two is the correct?

In our project, we currently use DbConnectionsMiddleware above AdminMiddleware (i.e. as currently shown in README.md) and still get the too many clients already error.