[FEATURE] Ability to subscribe while consuming

[Mephalich]

Hello, I'm trying to use the library in our applications and at some point I require to do a subscription "on-the-fly"...

Description:

• Stand-alone Redis 6.2.x server, C++17, several apps using redis++ to communicate each other and accept external commands
• Subscribe to a number of known channels when starting applications
• Use a dedicated thread for consume() jobs (try-consume-catch timeout-continue)
• Another thread decides to subscribe to an additional channel using existing Redis/subscriber

Issue:

• The subscription doesn't go through to the server until the consume reenters.

If I got it properly from the discussion threads there are just a couple ways of how this can be done at the moment:

1. Create the connection option with a non-zero socket_timeout. (cons) in this case there could be a possible case when we're missing incoming messages to the new channel due to a long timeout (cons) the thread will eat some resources if the timeout is too low
2. Publish a "poke" message to self at one of the already subscribed channel, so consume() will wake up and process writes/reads (cons) have an additional traffic and still have a chance to miss something due to communication latency

Ideas: -- to enrich the consume() with incoming/outgoing flags? -- to terminate the consume() by a request so the wheel can spin?

I suspect this goes down to the underlying hiredis api and I will need to redesign those temporary channels as streams or lists...

Thank you for any advice. Best regards!

[sewenew]

Create the connection option with a non-zero socket_timeout.

If you insist on Redis PubSub, this should be the best option so far.

Once you need to subscribe to a new channel, you can put the channel name to a queue, and when consume returns either by getting a message or timed out, the consuming thread subscribes new channels in the queue. Of course, you need to do thread synchronization. If you set timeout to be 10 - 100 milliseconds, I think it won't eat too much resource. You can do some benchmark test for it.

when we're missing incoming messages to the new channel due to a long timeout

Because Redis PubSub does not keep messages on server side, no matter how short the timeout is, there might be a time window between subscribing a channel and publishing a message (unless you can ensure the message will be published after you subscribe to the channel).

Publish a "poke" message to self at one of the already subscribed channel, so consume() will wake up and process writes/reads

If the connection is broken, the "poke" message will be lost forever. So this is not a reliable solution.

I suspect this goes down to the underlying hiredis api and I will need to redesign those temporary channels as streams or lists

Once the connection is blocked on consuming messages, you cannot send subscribe message to Redis until consume returns.

If you want a more reliable way for subscribe messages, I think Redis Stream is a better solution.

Regards

[Mephalich]

Sounds reasonable. Thanks for answers.

Best regards,