Extend message endpoint with parameters for producer and subscription

[MPSebastian]

User story

1. As a frontend-dev
2. I need parameters in the message endpoint
3. So that I can show the messages for producers and subscriptions in the UI

Acceptance criteria

• when the producer parameter is set, only {number} messages for this producer are returned
• when the subscription parameter is set, only {number} messages for this subscription are returned

Definition of done (DoD)

• Feature is done when the acceptance criteria are fullfiled
• Feature is done when there are test where it is necessary and they pass
• Feature is done when the linter passes
• Feature is done when it is merged onto our main branch (after the code has been reviewed by at least one other person)
• Feature is done when the code has been documentend or commented properly

DoD general criteria

• Feature has been fully implemented
• Feature has been merged into the mainline
• All acceptance criteria were met
• Product owner approved features
• All tests are passing
• Developers agreed to release

[nikkite99]

Producers and Subscriptions are a good idea but i don´t see any chance to get the messages by consumer :/

[phildree]

This is how we thought about this:

If it is possible to retrieve the subscriptions of the topic, then it should also be possible to retrieve the consumers for each subscription (using getConsumers() on the subscriptionStats).

This would mean, that it is possible to map consumers to subscriptions.

If the previously mentioned mapping is established it would be possible to iterate over the mapped subscriptions (for the consumer from the parameter) and retrieve the messages the same way as it would be done when querying for subscriptions directly. The messages of the so identified subscriptions would then have to be ordered in a single logical queue (by timestamp or sqn) and could then be traversed and retrieved.

The only part that we fought might be a problem, is to identify where the beginning of messages that have not yet been acknowledged is. We hope that the getUnackedMessages() method of the consumerStats is usable to move back in the queue and start at the acknowledged "slot".

If there are any issues you noticed, or things we did not think about, please let us know!

[nikkite99]

Your thoughts are great but in the solution you described every consumer has the same messages like its subscription. This is in some cases right but not in all. If you have a failover subscription your solution is not working i think. https://pulsar.apache.org/docs/3.0.x/concepts-messaging/#failover--partitioned-topics

[phildree]

Okay, you are absolutely right. If not all consumers receive the same messages then the proposed approach does not work. Therefore we will remove the functionality to retrieve the messages for the consumers. Thank you for your help! 😄