ed-pilots-network / backend

Project to consume the Kafka, process the messages and provide an API to access the data
Apache License 2.0
12 stars 5 forks source link

Databases Losing Connection #153

Closed Daniel-J-Mason closed 6 months ago

Daniel-J-Mason commented 6 months ago

Describe the bug After some period of time backend will lose connection with the databases and be unable to reconnect

To Reproduce Run stack, wait for modulith health to diminish, eventually loses connection

Additional context Speculated issue, IMC uses a pattern to check if a request exists, then insert. Due to the parallelism of our inserts this can stack up a series of identical requests when processed between that response window. As these conflicts collect, they slowly consume the full connection pool preventing any other database interaction