apache / pulsar-client-node

Apache Pulsar NodeJS Client
https://pulsar.apache.org/
Apache License 2.0
148 stars 85 forks source link

NodeJS Message router implementation #339

Open maximemg opened 1 year ago

maximemg commented 1 year ago

Hello,

My goal is to ensure a consumer will always receive key associated messages. But from my tests, on adding/removing consumers there are cases where Pulsar will do a rebalance.

So I think my best shot is to publish to a specific partition and subscribe to this specific partition with my customer.

So what I would do is implement the Message Router interface as explained here. But I could not make it in the Node.js client, as it does not seem to exist.

Will that be implemented in the future ?

Let me know if I'm mistaking or if it's unclear, as I'm pretty new to Apache Pulsar. Thanks for reading.

shibd commented 1 year ago

Yes, CustomMessageRoute is not supported yet.

maximemg commented 1 year ago

I was wondering if there's a way to ensure messages will be received by the same consumer in a key shared sub ? Because so far I could not achieve that with keys.

Is there any ETA for the CustomMessageRoute ?

shibd commented 1 year ago

I was wondering if there's a way to ensure messages will be received by the same consumer in a key shared sub ? Because so far I could not achieve that with keys.

  1. Consumers need to use KeyShared subscription type.
  2. and your producer need use KeyBased batchType (Node.js client is not yet implemented). Now you can disabled the producer batch first.
    // Create a producer and disabled batch
    const producer = await client.createProducer({
    topic: 'persistent://public/default/my-topic',
    sendTimeoutMs: 30000,
    batchingEnabled: false,    
    });
    // Create a consumer and use KeyShared sub type.
    const consumer = await client.subscribe({
    topic: 'persistent://public/default/my-topic',
    subscription: 'sub1',
    subscriptionType: 'KeyShared',
    ackTimeoutMs: 10000,
    });