The implementation of the Pulsar Pubsub component maintains the order of replicas in all subscribers however still round-robin delivers messages to all the replicas of the same app-id. In native Pulsar, all the messages with the same partition key are delivered to the same consumer when key_shared is set as the subscription type. The Dapr implementation today delivers the messages to all consumer replicas (same app-id) regardless of partition key, keeping the relative order but not limiting a partition key to a specific consumer replica.
This should be remedied to ensure consistency across the Dapr implementation of Pulsar pub/sub. The docs here also imply that it works the same.
Thanks @rochabr for the reproduction repo here, see screenshot below where the two subscribers with the same partition key (authoring-event-topic) and app id (subscriber) are on the bottom and the publisher is on the top. You can see the messages being delivered to two replicas instead of being pinned to one.
alicejgibbons
commented
8 hours ago
Describe the feature
The implementation of the Pulsar Pubsub component maintains the order of replicas in all subscribers however still round-robin delivers messages to all the replicas of the same app-id. In native Pulsar, all the messages with the same partition key are delivered to the same consumer when key_shared is set as the subscription type. The Dapr implementation today delivers the messages to all consumer replicas (same app-id) regardless of partition key, keeping the relative order but not limiting a partition key to a specific consumer replica.
This should be remedied to ensure consistency across the Dapr implementation of Pulsar pub/sub. The docs here also imply that it works the same.
Thanks @rochabr for the reproduction repo here, see screenshot below where the two subscribers with the same partition key (authoring-event-topic) and app id (subscriber) are on the bottom and the publisher is on the top. You can see the messages being delivered to two replicas instead of being pinned to one.
Release Note
RELEASE NOTE: FIX Pulsar key_shared functionality.