Feature Request: Extend the Pub/Sub API with delayed event scheduling

mmisztal1980 commented 3 years ago

In what area(s)?

/area runtime

Describe the feature

At this moment the Dapr pubsub's event publishing API supports publishing only, please consider extending this API with delayed event scheduling.

A user should be able to messages to a pubsub for delayed processing. Examples:

Scheduling a job to become available for processing by a system at a certain time.
Scheduling reminders/notifications
Improving resiliency by introducing exponential backoff after a service fails to process an event, while guaranteeing that the message is persisted by the pubsub backend

Requirements

This capability realizes a pubsub that supports a reliable distributed time-based scheduler, and the some of the pubsub backends support it already (there might be more, I'm not familiar with all supported backends):
When attempting to schedule an event on to an unsupported pubsub backend an exception should be thrown
Scheduled events should not be received by the subscribers until the defined enqueue time. Before that time, scheduled events should be cancellable. Cancellation should delete the event from the pubsub backend.

Release Note

RELEASE NOTE: N/A

mrchypark commented 3 years ago

How about creating a service that has a queue to publish to pubsub on a schedule that treat as state? This way can support the functionality for all pubsubs. And how about implement using virtual actor?

artursouza commented 3 years ago

This feature can be implemented using metadata when message is published and the scheduling information passed through to the component. A draft PR with this idea, or something else can be a good starter to evaluate how to address this.

There are other solutions too, like @mrchypark but Dapr can expose additional capabilities via metadata.

github-actions[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

dapr-bot commented 3 years ago

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as pinned, good first issue, help wanted or triaged/resolved. Thank you for your contributions.

Shiko1st commented 2 years ago

How about creating a service that has a queue to publish to pubsub on a schedule that treat as state? This way can support the functionality for all pubsubs. And how about implement using virtual actor?

Azure Service Bus supports this feature out-of-the-box, no need to invent a wheel, just a way to use ASB scheduling via dapr call...

geekgao commented 2 years ago

That's useful

itsoli91 commented 2 years ago

It's an important feature and already is supported by Azure Service Bus.

dapr-bot commented 2 years ago

This issue has been automatically marked as stale because it has not had activity in the last 60 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.

Shiko1st commented 2 years ago

Seems nobody cares...

yaron2 commented 2 years ago

Seems nobody cares...

Oh, folks care. Just 2 days ago I spoke with several end users who are very interested in this. I'll see if we can triage to 1.8 or 1.9.

calvin2021y commented 2 years ago

any update?

dapr-bot commented 2 years ago

This issue has been automatically marked as stale because it has not had activity in the last 60 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.

Shiko1st commented 2 years ago

@yaron2 any news?

eyeye commented 2 years ago

waiting for progress.

abhijeethaval commented 2 years ago

This will be very useful feature.

dapr-bot commented 2 years ago

This issue has been automatically marked as stale because it has not had activity in the last 60 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.

mmisztal1980 commented 2 years ago

no point in keeping this open

DataWorm commented 2 years ago

We are still interested in this feature. From our point of view, there is still a reason to keep it open! ;-)

Shiko1st commented 2 years ago

Yes, we are, but seems nobody cares )))

Shiko1st commented 2 years ago

no point in keeping this open

Is this feature now available?

yaron2 commented 2 years ago

Triaged into 1.10. I believe this should be doable. If anyone from the community interested in this feature wants to take the lead and help with design and implementation, that would greatly help.

yaron2 commented 1 year ago

Hi all, this feature is now at the proposal stage here: https://github.com/dapr/proposals/pull/12.

ytatipaka commented 1 year ago

We are having exactly same requirement for one of our dapr/pulsar usecase. As it is not available in current implementation, we might need to go with alternate approach for time being.

christcottrell commented 1 year ago

Just checking to see if there are plans to include this or this is being worked on. Is this project still going? Is DAPR viable? I can't even get the demo for pub sub to work and raised an issue for it and no response at all. Weeks before it was working, but seems like some change broke it.

clintsinger commented 1 year ago

I was thinking that it would be great to have the Actor Reminder feature outside of actors. Sometimes you just need to schedule an event that fires at some point in the future. Building it into the pubsub system would be a great alternative without having to go through all of the boilerplate for setting up an actor for just this purpose.

yaron2 commented 1 year ago

I was thinking that it would be great to have the Actor Reminder feature outside of actors. Sometimes you just need to schedule an event that fires at some point in the future. Building it into the pubsub system would be a great alternative without having to go through all of the boilerplate for setting up an actor for just this purpose.

This is being designed

dapr / dapr