Closed egekocabas closed 9 months ago
current status: β
Overall Coverage
Lines Covered Coverage Threshold Status 2046 1500 73% 0% π’ New Files
No new covered files...
Modified Files
File Coverage Status parma_analytics/db/prod/engine.py 100% π’ TOTAL 100% π’ updated for commit:
4f44750
by actionπ
Motivation
We were getting the database connection pool error while we were bursting one endpoint that uses the database connection.
I searched about this error, and it seems that creating an engine for every time (every time
get_engine()
orget_session()
is called) consumes all the connection pools.Using a singleton pattern for the engine object and letting it manage its connection pool fixed the problem locally. I managed to send requests (uses the
get_session()
) to the/feed-raw-data
endpoint with 5000 requests in a loop without a connection pool problem.Changes
get_engine()
updated to use a single engine object. Pre-commit warning is suppressed because if we implement a class and then use its variable then we need to change every place where we useget_engine()
andget_session()
. I suggest to use it like that and then if it works and we have a time we can change the structure in here.Checklist
!
(e.g.feat!: Update endpoint
)