gdaws / stompit

STOMP client library for node.js
MIT License
163 stars 36 forks source link

Bulk processing #111

Closed Dadibom closed 3 years ago

Dadibom commented 4 years ago

In many cases, processing batches of messages instead of processing them one at a time can provide huge performance benefits, such as multiple database inserts being able to be wrapped in a single transaction.

Having a simple subscribe method that provides a list of messages in the callback whenever there's multiple received messages at once would be great.

gdaws commented 3 years ago

Bulk processing of messages can be achieved at the application level. Open a subscription with client ack mode and when a message is received push it onto a local queue for the next processing batch. After a batch has been processed send an acknowledgement for the last message processed. The client sees the performance benefit of sending one ack for many messages and the server could potentially perform ack handling of many messages as a single transaction.

I'm not sure it's possible for stompit, by its design, to deliver a list of messages to a subscription callback. How can it know when multiple messages are received at once? The library only reads a frame header then control is passed to the subscription callback where that reads the frame body. It's for the application to read a message and then either process it immediately or defer it.

Dadibom commented 3 years ago

Thanks @gdaws, I don't even know if stompit polls or gets data pushed to it so I have no idea how it would be done internally. I'll just write a simple wrapper function and use that as a handler.