search

mdl29 / donkeycarLPH

Front + backend to manage donkeycars during demonstrations events
4 stars 8 forks source link

[backend] Increase database connexion pool #219

Closed Benvii closed 2 years ago

Benvii commented 2 years ago

Error : 

2022-09-17 17:18:09,903 INFO:     uvicorn.access: 192.168.20.40:44666 - "GET /cars/?skip=0&limit=10 HTTP/1.1" 200
2022-09-17 17:18:09,905 INFO:     uvicorn.access: 192.168.20.40:44700 - "GET /cars/?skip=0&limit=4 HTTP/1.1" 200
2022-09-17 17:18:09,927 ERROR:    uvicorn.error: Exception in ASGI application
Traceback (most recent call last):
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/uvicorn-0.17.6-py3.8.egg/uvicorn/protocols/http/h11_impl.py", line 366, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/uvicorn-0.17.6-py3.8.egg/uvicorn/middleware/proxy_headers.py", line 75, in __call__
    return await self.app(scope, receive, send)
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/fastapi-0.66.0-py3.8.egg/fastapi/applications.py", line 199, in __call__
    await super().__call__(scope, receive, send)
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/starlette-0.14.2-py3.8.egg/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/starlette-0.14.2-py3.8.egg/starlette/middleware/errors.py", line 181, in __call__
    raise exc from None
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/starlette-0.14.2-py3.8.egg/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/starlette-0.14.2-py3.8.egg/starlette/middleware/cors.py", line 86, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/starlette-0.14.2-py3.8.egg/starlette/middleware/cors.py", line 142, in simple_response
    await self.app(scope, receive, send)
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/starlette-0.14.2-py3.8.egg/starlette/exceptions.py", line 82, in __call__
    raise exc from None
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/starlette-0.14.2-py3.8.egg/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/starlette-0.14.2-py3.8.egg/starlette/routing.py", line 580, in __call__
    await route.handle(scope, receive, send)
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/starlette-0.14.2-py3.8.egg/starlette/routing.py", line 241, in handle
    await self.app(scope, receive, send)
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/starlette-0.14.2-py3.8.egg/starlette/routing.py", line 52, in app
    response = await func(request)
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/fastapi-0.66.0-py3.8.egg/fastapi/routing.py", line 216, in app
    raw_response = await run_endpoint_function(
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/fastapi-0.66.0-py3.8.egg/fastapi/routing.py", line 151, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/starlette-0.14.2-py3.8.egg/starlette/concurrency.py", line 40, in run_in_threadpool
    return await loop.run_in_executor(None, func, *args)
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/mdl29/donkeycarLPH/backend/donkeycarmanager/routers/cars.py", line 27, in read_cars
    return crud.get_cars(db, skip=skip, limit=limit)
  File "/home/mdl29/donkeycarLPH/backend/donkeycarmanager/crud/cars.py", line 15, in get_cars
    return db.query(models.Car).offset(skip).limit(limit).all()
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/SQLAlchemy-1.4.36-py3.8-linux-x86_64.egg/sqlalchemy/orm/query.py", line 2768, in all
    return self._iter().all()
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/SQLAlchemy-1.4.36-py3.8-linux-x86_64.egg/sqlalchemy/orm/query.py", line 2903, in _iter
    result = self.session.execute(
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/SQLAlchemy-1.4.36-py3.8-linux-x86_64.egg/sqlalchemy/orm/session.py", line 1695, in execute
    conn = self._connection_for_bind(bind)
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/SQLAlchemy-1.4.36-py3.8-linux-x86_64.egg/sqlalchemy/orm/session.py", line 1536, in _connection_for_bind
    return self._transaction._connection_for_bind(
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/SQLAlchemy-1.4.36-py3.8-linux-x86_64.egg/sqlalchemy/orm/session.py", line 747, in _connection_for_bind
    conn = bind.connect()
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/SQLAlchemy-1.4.36-py3.8-linux-x86_64.egg/sqlalchemy/engine/base.py", line 3234, in connect
    return self._connection_cls(self, close_with_result=close_with_result)
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/SQLAlchemy-1.4.36-py3.8-linux-x86_64.egg/sqlalchemy/engine/base.py", line 96, in __init__
    else engine.raw_connection()
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/SQLAlchemy-1.4.36-py3.8-linux-x86_64.egg/sqlalchemy/engine/base.py", line 3313, in raw_connection
    return self._wrap_pool_connect(self.pool.connect, _connection)
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/SQLAlchemy-1.4.36-py3.8-linux-x86_64.egg/sqlalchemy/engine/base.py", line 3280, in _wrap_pool_connect
    return fn()
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/SQLAlchemy-1.4.36-py3.8-linux-x86_64.egg/sqlalchemy/pool/base.py", line 310, in connect
    return _ConnectionFairy._checkout(self)
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/SQLAlchemy-1.4.36-py3.8-linux-x86_64.egg/sqlalchemy/pool/base.py", line 868, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/SQLAlchemy-1.4.36-py3.8-linux-x86_64.egg/sqlalchemy/pool/base.py", line 476, in checkout
    rec = pool._do_get()
  File "/home/mdl29/donkeycarLPH/backend/.venv/lib/python3.8/site-packages/SQLAlchemy-1.4.36-py3.8-linux-x86_64.egg/sqlalchemy/pool/impl.py", line 134, in _do_get
    raise exc.TimeoutError(
sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30.00 (Background on this error at: https://sqlalche.me/e/14/3o7r)

Solution : https://stackoverflow.com/questions/26100925/how-to-increase-poolsize-in-sqlalchemy

We should also optimize the frontend connections managment.

yannis-mlgrn commented 2 years ago

i've optimized the dashboard in #214