etf1 / kafka-message-scheduler

scheduler for kafka messages
MIT License
76 stars 14 forks source link

How to set partition ? #36

Closed yusuf5033 closed 1 year ago

yusuf5033 commented 2 years ago

in a Kafka Producer (for example nodejs - kafka-node), in addition the topic, We can set the partition number as well for the message.

Like stated in the docs, we can set the topic: scheduler-target-topic: online-videos

Lets say we want to send the message to "online videos" and partition : 4 Can we set that in the header also ?

fkarakas commented 2 years ago

hello @yusuf5033 , Thank you for your message. Today it is not possible to set the target-partition, but i think it can be done easily in the scheduler-target-topic like so : scheduler-target-topic: online-videos:4 where 4 is the target partition.

WDYT ?

yusuf5033 commented 2 years ago

Hello, just to be clear, when you say ".. i think it can be done easily ..." means after you implemented it right ? :) I think that would be a great feature.

fkarakas commented 2 years ago

;) yes For my information, how to plan to use the scheduler ? user use case are always an interesting feedback :)

yusuf5033 commented 2 years ago

Thank you,

We are switching to Kafka to deliver our bulk push notifications to user's mobile apps. In our admin panel, customers can send bulk notifications to their end users. We developed the the part where they can send messages right away, but we also want to give them ability to schedule their messages to a future time. For that feature kafka-message-scheduler seems really a good solution.

But we use different topic partitions for each "notification type" so we are kind of stuck in that part. I can use yet another topic for just deliver "scheduled" messages to actual topic / partitions but it is going to make things a bit longer.

ldechoux commented 2 years ago

Hello @yusuf5033,

To be sure to correctly undestand. Why did you use partition to identify each notification type ? It seems more simple to use different topic for each notification type. What do you think ?