dataforgoodfr / 12_bloom

23 stars 11 forks source link

API Endpoint segments => erreur 500 #194

Closed rv2931 closed 2 months ago

rv2931 commented 2 months ago

@ComeTiss : Petit erreur 500 quand j'essaye de get les segments:


bloom_backend    | ERROR:    Exception in ASGI application
bloom_backend    | Traceback (most recent call last):
bloom_backend    |   File "/venv/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 399, in run_asgi
bloom_backend    |     result = await app(  # type: ignore[func-returns-value]
bloom_backend    |   File "/venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
bloom_backend    |     return await self.app(scope, receive, send)
bloom_backend    |   File "/venv/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
bloom_backend    |     await super().__call__(scope, receive, send)
bloom_backend    |   File "/venv/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
bloom_backend    |     await self.middleware_stack(scope, receive, send)
bloom_backend    |   File "/venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
bloom_backend    |     raise exc
bloom_backend    |   File "/venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
bloom_backend    |     await self.app(scope, receive, _send)
bloom_backend    |   File "/venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
bloom_backend    |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
bloom_backend    |   File "/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
bloom_backend    |     raise exc
bloom_backend    |   File "/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
bloom_backend    |     await app(scope, receive, sender)
bloom_backend    |   File "/venv/lib/python3.10/site-packages/starlette/routing.py", line 756, in __call__
bloom_backend    |     await self.middleware_stack(scope, receive, send)
bloom_backend    |   File "/venv/lib/python3.10/site-packages/starlette/routing.py", line 776, in app
bloom_backend    |     await route.handle(scope, receive, send)
bloom_backend    |   File "/venv/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle
bloom_backend    |     await self.app(scope, receive, send)
bloom_backend    |   File "/venv/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
bloom_backend    |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
bloom_backend    |   File "/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
bloom_backend    |     raise exc
bloom_backend    |   File "/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
bloom_backend    |     await app(scope, receive, sender)
bloom_backend    |   File "/venv/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
bloom_backend    |     response = await func(request)
bloom_backend    |   File "/venv/lib/python3.10/site-packages/fastapi/routing.py", line 307, in app
bloom_backend    |     response = actual_response_class(content, **response_args)
bloom_backend    |   File "/venv/lib/python3.10/site-packages/starlette/responses.py", line 184, in __init__
bloom_backend    |     super().__init__(content, status_code, headers, media_type, background)
bloom_backend    |   File "/venv/lib/python3.10/site-packages/starlette/responses.py", line 41, in __init__
bloom_backend    |     self.body = self.render(content)
bloom_backend    |   File "/venv/lib/python3.10/site-packages/starlette/responses.py", line 187, in render
bloom_backend    |     return json.dumps(
bloom_backend    |   File "/usr/local/lib/python3.10/json/__init__.py", line 238, in dumps
bloom_backend    |     **kw).encode(obj)
bloom_backend    |   File "/usr/local/lib/python3.10/json/encoder.py", line 199, in encode
bloom_backend    |     chunks = self.iterencode(o, _one_shot=True)
bloom_backend    |   File "/usr/local/lib/python3.10/json/encoder.py", line 257, in iterencode
bloom_backend    |     return _iterencode(o, 0)
bloom_backend    | ValueError: Out of range float values are not JSON compliant```
rv2931 commented 2 months ago

@ComeTiss Tu saurais me dire si cette erreur est systématique sur tous les vessels/excursions ou seulement sur certain/es ? tu pourrais me donner quelques requêtes et/ou vessel_id/excursion_id qui pose souci pour la récup de segments ? Une hypothèse est qu'il y a des données null/NaN quelque part en base et que le json_encoder ne les accepte pas... mais c'est qu'une hypothèse à ce stade

rv2931 commented 2 months ago

C'est bon je le reproduis avec la requête : http://localhost:8000/vessels/1160/excursions/181/segments

rv2931 commented 2 months ago

Je pense que j'ai trouvé le problème Certains segments contiennent des valeurs "NaN" issues de pandas j'imagine sauf qu'elle devraient être mise à NULL et non rester en NaN le problème doivent venir de la partie ETL @njouanin tu confirmes ? Le contournement consiste à remplacer les valeurs NaN par NULL en attente d'avoir des valeurs valides côté ETL La requête SQL de contournement:

update fct_segment
set average_speed = null
where average_speed = 'NaN';

image

njouanin commented 2 months ago

Corrigé.