search

52North / pygeoapi-odc-provider

Provider plugin for pygeoapi to include Open Data Cube as a data resource.
Apache License 2.0
4 stars 1 forks source link

HTTP 500 responses when doing multiple (parallel) coverage requests #10

Open MartinPontius opened 2 years ago

MartinPontius commented 2 years ago

When multiple (parallel) coverage requests are coming in the server returns error code 500 as it fails with

...
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) FATAL:  sorry, too many clients already
(Background on this error at: https://sqlalche.me/e/14/e3q8)

Possibly multiple database connections are opened but not closed again or postgres restricts the number of connections per default too much for our use case.

jerstlouis commented 2 years ago

My suggestion would be to not necessarily require more connections to postgres (though that could also be increased), but to implement a queue where a client can be put on hold until a DB connection becomes available again without having to return a 500 error. Perhaps this could also be implemented higher up, maybe even above the pygeoapi provider level?

MartinPontius commented 2 years ago

Update: this problem seems constantly to occur for simple/single requests after the service has been running for a while.