Open MPSebastian opened 1 year ago
Producers and Subscriptions are a good idea but i don´t see any chance to get the messages by consumer :/
This is how we thought about this:
If it is possible to retrieve the subscriptions of the topic, then it should also be possible to retrieve the consumers for each subscription (using getConsumers() on the subscriptionStats).
This would mean, that it is possible to map consumers to subscriptions.
If the previously mentioned mapping is established it would be possible to iterate over the mapped subscriptions (for the consumer from the parameter) and retrieve the messages the same way as it would be done when querying for subscriptions directly. The messages of the so identified subscriptions would then have to be ordered in a single logical queue (by timestamp or sqn) and could then be traversed and retrieved.
The only part that we fought might be a problem, is to identify where the beginning of messages that have not yet been acknowledged is. We hope that the getUnackedMessages() method of the consumerStats is usable to move back in the queue and start at the acknowledged "slot".
If there are any issues you noticed, or things we did not think about, please let us know!
Your thoughts are great but in the solution you described every consumer has the same messages like its subscription. This is in some cases right but not in all. If you have a failover subscription your solution is not working i think. https://pulsar.apache.org/docs/3.0.x/concepts-messaging/#failover--partitioned-topics
Okay, you are absolutely right. If not all consumers receive the same messages then the proposed approach does not work. Therefore we will remove the functionality to retrieve the messages for the consumers. Thank you for your help! 😄
User story
Acceptance criteria
Definition of done (DoD)
DoD general criteria