afair / postgresql_cursor

ActiveRecord PostgreSQL Adapter extension for using a cursor to return a large result set
MIT License
599 stars 48 forks source link

PG::ConnectionBad: PQsocket() can't get socket descriptor: ROLLBACK #64

Open ajparise opened 1 year ago

ajparise commented 1 year ago

Our app opens a cursor which can run for hours at a time. On the longer runs (7+ hours) the cursor occasionally errors out with:

ActiveRecord::StatementInvalid
PG::ConnectionBad: PQsocket() can't get socket descriptor: ROLLBACK

I am trying to rule this gem out as the culprit. I'm not sure if this is masking a different error. Should we be passing with_hold: true in the options?

Here is a snippet of the cursor loop:

# BATCH_SIZE = 10_000
 records.each_instance(block_size: BATCH_SIZE) { |record| @handler.call(record) }

Couple things that may or may not be pertinent:

guemidiborhane commented 1 year ago

+1 we're having the same issue but only in production.

guemidiborhane commented 1 year ago

After some investigations, seems like the issue in our case is related to the query being run against replica postgresql server, fixed by setting this 2 config lines in postgresql.conf:

max_standby_archive_delay = 600s 
max_standby_streaming_delay = 600s