search

vapor / postgres-nio

🐘 Non-blocking, event-driven Swift client for PostgreSQL.
https://api.vapor.codes/postgresnio/documentation/postgresnio/
MIT License
305 stars 70 forks source link

Double ping pong that crashes the app. #436

Closed notcome closed 8 months ago

notcome commented 8 months ago

Describe the bug

So this is a follow up to my previous issue #435 . I mentioned that I often ran into crash on the following line:

https://github.com/vapor/postgres-nio/blob/c8269926eb3b705b70aff1975860e357760123c8/Sources/ConnectionPoolModule/PoolStateMachine%2BConnectionGroup.swift#L441

So I have decided to to modify the source code a bit, added some log, and observed the following thing:

  1. availableStreams will reduce 1 before ping pong, and increase back afterwards;
  2. when the issue happens, availableStreams is already at 0;
  3. the issue only happens if, according to the log, two ping pong happens in vert short interval.

So I suppose this is just a strange weird data race around keep alive timers. But I have seen no pattern why this might happen, nor can I reliably reproduce this issue. Plus I suppose we will never see it if the ping pong can finish in very short period.

I wonder if there is anything I can do to help you people track this down. I am a bit occupied right now, but I will try to add some log and data race check on a local fork when I get some free time.

Expected behavior No error.

Environment Vapor Framework version: 4.84.6 Vapor Toolbox version: 18.7.4 OS version: macOS 14.0 (23A344)

fabianfett commented 8 months ago

@notcome Thanks I'll look into this. As a workaround you can disable keep-alives for now:

clientConfiguration.options.keepAliveBehavior = nil
fabianfett commented 8 months ago

Released in 1.19.1. Please check if the issue is fixed for you now.