Should apply all messages from ShapeStream transactionally (even though with chunking you cannot guarantee that the batch of messages received form a transaction) - previously was using one transaction per message and blocking the main loop as well!
We could potentially remove the transaction altogether but I'm assuming that pg might optimize writing many messages together when committing as a transaction either way.
I've also awaited each transaction to finish as the new version of ShapeStream will be able to apply backpressure based on the callback provided to process messages, and avoid flooding PG
Should apply all messages from
ShapeStream
transactionally (even though with chunking you cannot guarantee that the batch of messages received form a transaction) - previously was using one transaction per message and blocking the main loop as well!We could potentially remove the transaction altogether but I'm assuming that pg might optimize writing many messages together when committing as a transaction either way.
I've also awaited each transaction to finish as the new version of
ShapeStream
will be able to apply backpressure based on the callback provided to process messages, and avoid flooding PG