Support to set the last delivered entry for Redis Stream Pubsub

[zhangcz828]

Describe the feature

Currently redis stream pubsub use "0" as the last delivered entry ID in the stream, here is the related code. So this means it hardcode the consumer MUST receive messages in the specified stream from the very first message.

There are some sceneries that the consumer want to specify the ID of the last entry in the stream to start to sync from, for example to set the $ but not 0 for the ID to ask sync from the last ID of the entry in the stream.

The request is to add one more config parameter in the component of type pubsub.redis to allow the human to setup the ID of the last entry in the stream to sync from.

Release Note

ADD one more parameter in the pubsub.redis config to support setup the ID of the last entry in the stream.

RELEASE NOTE:

[berndverst]

Would it be ok to configure this via the component metadata? That means it will read once when the component starts up (or if it restarts)?

[berndverst]

I'm proposing a new metadata property pubsubLastDeliveredEntry with default value of 0. If that meets your requirements we can easily add this for Dapr 1.13.

[berndverst]

A new pubsubLastDeliveredEntry pubsub property of type *string has to be added here: https://github.com/dapr/components-contrib/blob/main/common/component/redis/settings.go#L85 It's important to add the mdonly:"pubsub" tag.

The entry should look like so:

type Settings struct {
  PubsubLastDeliveredEntry *string `mapstructure:"pubsubLastDeliveredEntry" mdonly:"pubsub"`
}

All that remains then is to update: https://github.com/dapr/components-contrib/blob/main/pubsub/redis/redis.go#L113

The last parameter should remain "0" if r.clientSettings.pubsubLastDeliveredEntry is nil. Otherwise, it should be *r.clientSettings.pubsubLastDeliveredEntry.

And lastly the new property needs to be added here: https://github.com/dapr/components-contrib/blob/main/pubsub/redis/metadata.yaml

And after all of that we need to open a documentation PR also.

[sadath-12]

/assign

[berndverst]

There needs to be a certification test for this - and that could be a bit harder to achieve.

Essentially what we need is: Publish a bunch of pubsub events in order - ideally numbered sequentially. Then connect with a different sidecar and different component definition where the lastdelivered item is configured to be something that is not 0, for example 6.

Then you have to check that the items 1-5 were indeed not delivered as expected.

[zhangcz828]

I'm proposing a new metadata property pubsubLastDeliveredEntry with default value of 0. If that meets your requirements we can easily add this for Dapr 1.13.

Thanks @berndverst , this is what I need. And also thanks to @sadath-12 for the PR, looking forward this to move forward