database connection issues

GIScience / ohsome-api

API for analysing OpenStreetMap history data

https://api.ohsome.org

GNU Affero General Public License v3.0

47 stars 8 forks source link

database connection issues #167

Open joker234 opened 3 years ago

joker234 commented 3 years ago

The ohsome API should handle database connection issues better. Currently, if there is a network hickup or the database restarts for some reason, the ohsome API doesn't notice that. It just returns errors if the database is used. I would even consider this as a bug, feel free to change the labelling if you don't agree.

I think there are multiple possibilities to improve the database-connection behaviour:

Let the ohsome API fail after a timeout in the database-connection and let the service handler (e.g. systemd) take care of a restart.
Improve connection handling in the ohsome API. Especially, reconnect automatically and don't return errors like this if a connection is possible again: java.lang.RuntimeException: The timerange metadata could not be retrieved from the db.

FabiKo117 commented 3 years ago

I rather see this as an enhancement, as the functionality of dealing with a failing DB connection is just not there yet. You can argue though that the state in which the ohsome API goes after such an incidence is somewhat 'buggy'.

Improve connection handling in the ohsome API. Especially, reconnect automatically and don't return errors like this if a connection is possible again: java.lang.RuntimeException: The timerange metadata could not be retrieved from the db.

This error message is a cause of a failing connection though, so that's something good already meaning the API does not just fail when the DB is gone. I agree it should try to reconnect automatically then, but for that we'd have to implement also an automatic check to see if the DB connection is still there and working, right?

rtroilo commented 3 years ago

Basically using a ConnectionPool (e.g. Hikari) would already solve the problem, as it trys to reconnect if the connection is closed! Also have one connection per thread is also advisable and would fix the problem with the IgniteThinDriver which is not multithreaded!

tyrasd commented 3 years ago

in the long term, https://github.com/GIScience/oshdb/issues/225 might become relevant

FabiKo117 commented 3 years ago

also related issue: #37

bonaparten commented 3 years ago

java.lang.RuntimeException: The timerange metadata could not be retrieved from the db.
Regarding this error, avoiding to extract metadata for every request should solve the problem.