search

apache / pulsar-client-node

Apache Pulsar NodeJS Client
https://pulsar.apache.org/
Apache License 2.0
147 stars 85 forks source link

[Feature request] Configure the auto-created DLQ producer #366

Closed dreamerblue closed 6 months ago

dreamerblue commented 7 months ago

We really like pulsar and would like to see more features in the node client.

We found the deadLetterPolicy only provides a few options, and the auto-created producer is non-configurable. We are requesting to add an extra producer config field under ConsumerConfig.deadLetterPolicy that will allow auto-created producers to configure properties such as the maximum number of batching messages, maximum delay to publish and whether to enable batching, etc.

dreamerblue commented 7 months ago

Another reason we need this feature:

If we send multiple dead letter messages in a batch then they will have the same ORIGIN_MESSAGE_ID (see the demo below). this is a big difficulty to manually troubleshoot the original message from DLQ. So we need to turn off batching.

image
shibd commented 7 months ago

The producer of DLQ should be a low-frequency operation, and we can turn off batch sending by default, just like the Java client behaves. And there's no need to expose those configurations, what do you think?

https://github.com/apache/pulsar/blob/252509e08018eeed189e0d5492c3664a67bced06/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java#L2166-L2189

dreamerblue commented 7 months ago

Thanks for your reply. I think it's reasonable. The new client behavior can avoid some potential troubles.

shibd commented 6 months ago

Close this issue, it solves by cpp3.5.0, and the node.js client has already upgraded it.