Open JunFu0814 opened 2 years ago
there is already another PIP with this PIP-number: PIP-156: Build and Run Pulsar Server on Java 17 https://github.com/apache/pulsar/issues/15207
please see https://github.com/apache/pulsar/wiki/Pulsar-Improvement-Proposal-%28PIP%29
sorry for bothering again, just upcounting 1 doesn't do the trick: There is already PIP 157, see PIP-157: Bucketing topic metadata to allow more topics per namespace https://github.com/apache/pulsar/issues/15254 @lhotari could you help?
The issue had no activity for 30 days, mark with Stale label.
it looks like addressed by https://github.com/apache/pulsar/issues/16763
Motivation
Business scene
Delayed messages are a common scenario in the Message business system. For example, group-opening reminders in group buying activities, automatic deductions when continuous monthly subscriptions expire, and coupon expiration reminders can all be implemented with delayed messages.
Current Mode
Currently Pulsar implements arbitrarily delayed messages based on in-memory time rounds. The Producer side can send a mixture of ordinary messages and delayed messages to topics and persist them to Bookkeeper. Now delayed messages can only support shard mode subscriptions,each subscription group will identify delayed messages and add them to the time wheel. The default time granularity of the time wheel is 1 second. The time wheel stores the delayed timestamp, ledgerId and EntryId related index information. When the time arrives, the message will be read and sent to the Consumer according to the ledgerId and EntryId.
Current Problems
Goat
Optimization ideas
Reduce the delay message magnitude for the current build time round. We can use the delay class to divide the overall delayed messages into time ranges, so that we only need to care about the most recent delayed messages. For example, a message with a relatively large time span is stored in a common topic (delay level topic), and then a certain mechanism is used to ensure that the message that is about to expire is placed in the time wheel over time, so that to a certain extent, the message can be greatly improved. Reduce the magnitude of delayed messages on the time wheel, thereby mitigating the impact of the problems described above.
Implementation details
Questions
Proposed Changes
Client API
#delayLevelTopicEnabled
#delayLevelTopicName
#partitionStrategy
inProducerConfigurationData.java
DelayLevelTopicPartitionStrategy.java
#enableDelayLevelTopic(boolean delayLevelTopicEnabled)
#delayLevelTopic(String delayLevelTopicName)
#delayLevelTopicPartitionStrategy(DelayLevelTopicPartitionStrategy strategy)
inProducerBuilder.java
Broker
TODO
Functions
TODO : Timing synchronization level delay message in topic to business topic