apache / pulsar

Apache Pulsar - distributed pub-sub messaging system
https://pulsar.apache.org/
Apache License 2.0
14.15k stars 3.57k forks source link

[admin] enhance rest api to help calculate the ending MessageID with a starting MessageID and an offset number #13160

Open nlu90 opened 2 years ago

nlu90 commented 2 years ago

Is your feature request related to a problem? Please describe. Data processing framework sometimes need to limit the # of messages read from one read iteration, and this requires knowing the ending message id. Unlike kafka, pulsar's message id is not linear offset based, so this information is tricky to get on client side.

Describe the solution you'd like Add a new admin rest api or enhance the existing topics get-message-id cmd that will help calculate the messageID given an existing messageID and an offset. ./bin/pulsar-admin topics get-message-id --staringID $STARTING_MESSAGE_ID --count $N

the command should return an message id that is the $N th message starting from the $STARTING_MESSAGE_ID

And we could reuse some of the logic here: https://github.com/apache/pulsar/blob/e7b0e3dbc5b6a05e955dc2cad034de0487463150/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java#L3155

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

github-actions[bot] commented 2 years ago

The issue had no activity for 30 days, mark with Stale label.

RedemptionC commented 2 years ago

Hi, I'd like to work on this issue,maybe you could assign this to me

github-actions[bot] commented 2 years ago

The issue had no activity for 30 days, mark with Stale label.