search

cody-greene / node-rabbitmq-client

RabbitMQ (0-9-1) client library with auto-reconnect & zero dependencies
MIT License
130 stars 9 forks source link

Streaming/batched publishers #39

Open cody-greene opened 9 months ago

cody-greene commented 9 months ago

In certain cases I'd like to rapidly publish a lot of messages at once, with confirms enabled. Currently this means collecting a bunch of promises and using Promise.all(...). It would be nice if I had a API conforming to the node WritableStream interface (where too many unconfirmed messages creates back-pressure), or maybe just a sendAll(messages) method.

dermasmid commented 6 months ago

is there something built in to the spec for batch publishing?

cody-greene commented 6 months ago

There are some aspects. Transaction-mode exists, but it's mutually exclusive with publisher confirms so I don't recommend it. A single BasicAck from the exchange can (theoretically) confirm multiple messages from the client so just sending everything as fast as possible seems to be the way to go.

Also thinking about this more, the WritableStream interface does not seem a good match for this library. Node streams are fragile (a single error results in a closed stream) but this is designed around recovery, retrying, etc. I think the best option is pub.sendAll(messages)