search

kibae / pg-logical-replication

PostgreSQL Logical Replication client for node.js
https://www.npmjs.com/package/pg-logical-replication
MIT License
95 stars 22 forks source link

Error: Connection terminated #2

Closed super-cache-money closed 2 years ago

super-cache-money commented 6 years ago

Firstly, thank you very much for making this library. I can't believe more people aren't capitalising on logical replication.

We have gotten your example working, but we keep getting the following errors, with pg@6.2.2:

Trace: Logical replication initialize error Error: Connection terminated
  at Connection.<anonymous> (/app/dir/node_modules/pg-logical-replication/node_modules/pg/lib/client.js:200:29)
  at Connection.g (events.js:291:16)
  at emitNone (events.js:86:13)
  at Connection.emit (events.js:185:7)
  at Socket.<anonymous> (/app/dir/node_modules/pg-logical-replication/node_modules/pg/lib/connection.js:141:10)
  at emitNone (events.js:91:20)
  at Socket.emit (events.js:185:7)

This error keeps throwing around once a minute, but the streaming still works and I can still see the DB changes.

Edit: The same occurs when we use pg@7.4.0.

jmealo commented 6 years ago

Kudos on this library. I'm currently wrapping pg_recvlogical, but it's difficult to build that out of the pg source tree, so it limits users to running it on a system with postgrestsql installed.

See pg_recvlogical wrapped: https://github.com/JarvusInnovations/lapidus

I find the re-connection behavior of pg_recvlogical to be correct as Lapidus has been running for years in production without issue. Perhaps if I can add additional tests (that will fail if I switch to pg-logical-replication) to verify the connection handling I can provide some feedback.

The behavior I like here is if either process dies we don't continue to acknowledge/advanced the LSN position for the slot.

kibae commented 6 years ago

Hello, @super-cache-money

Check the server log to see if there are any messages below. LOG: terminating walsender process due to replication timeout

Check out this document. https://www.postgresql.org/docs/current/static/runtime-config-replication.html#GUC-WAL-SENDER-TIMEOUT

Could you notify me if the same problem occurs after changing the wal_sender_time value?

bopjesvla commented 6 years ago

I have the same problem, but the stream only keeps working when using the proc function provided in the example, which handles automatic restarting. Setting wal_sender_timeout to 0 fixes the problem, but doesn't that mess up disconnection detection?

c4l3b commented 5 years ago

I have submitted a PR that should resolve this issue https://github.com/kibae/pg-logical-replication/pull/8