jackc / pglogrepl

PostgreSQL logical replication library for Go.
MIT License
319 stars 59 forks source link

Only receive one record one time? #41

Closed zhanghaiyang9999 closed 1 year ago

zhanghaiyang9999 commented 1 year ago

HI @jackc, does the following code just can receive one changed record each time?

rawMsg, err := t.conn.ReceiveMessage(ctx)

If so, is there some performance issues if the changes are frequently(see add lots of table records in a short time)? thanks!

diabloneo commented 1 year ago

The message sent through Postgres logical replication protocol by pgoutput plugin carries more than record data.

For logical replication related protocols, following documents describe the protocol:

  1. https://www.postgresql.org/docs/current/protocol-replication.html
  2. https://www.postgresql.org/docs/current/protocol-logicalrep-message-formats.html

If t.conn.ReceiveMessage() returns a CopyData message, it may contains record data represent as WAL data. Event if the data received is a XLogData, it may be something other the recorded being updated.

zhanghaiyang9999 commented 1 year ago

The message sent through Postgres logical replication protocol by pgoutput plugin carries more than record data.

For logical replication related protocols, following documents describe the protocol:

  1. https://www.postgresql.org/docs/current/protocol-replication.html
  2. https://www.postgresql.org/docs/current/protocol-logicalrep-message-formats.html

If t.conn.ReceiveMessage() returns a CopyData message, it may contains record data represent as WAL data. Event if the data received is a XLogData, it may be something other the recorded being updated.

Thanks @diabloneo ,From the example https://github.com/jackc/pglogrepl/blob/master/example/pglogrepl_demo/main.go The example seems that only can handle one record(insert ,update or delete) every time, and for *pglogrepl.InsertMessage, the col.DataType has 3 cases, one is 'u', the others are 'n' and 't', I can understand the 'n' and 't' types for the insertMesage, but for 'u' , I don't know when will it happen. Thanks @diabloneo