Push messages to multiple subscribers

[5lava]

Hello. First of all, thank you for your work.

I often need to send the same message to multiple subscribers. Currently I open regular HTTP keep-alive connection and perform multiple /pub requests. Tens, sometimes hundreds. Obviously doing that by sending numerous HTTP requests is not optimal solution. It would be great if the nginx-push-stream-module could allow sending messages to multiple recipients, something like "/pub?ids=ch1,ch14,ch20,ch35".

Please let me know if this feature is planned. Thank you!

[wandenberg]

Yes, it's planned publish the same message to multiple channels at once.

[5lava]

While developing our application we realized that we almost never send exactly the same message to multiple clients. Unfortunately each message differs a little bit (contains some client-specific information), so we have to perform 10 cURL requests to send those messages to 10 clients. We're sending many messages and even with persistent connections that's pretty stressful for both client and nginx.

Long story short, I'd like to propose a feature request - push multiple messages to multiple clients in one request. To do that we need to split POST data to multiple messages, that can be done by specifying message lengths (similarly like push_stream_channels_path allows to specify number of backtrack messages). For example, to send 3 messages we would specify 3 channel ids and 3 message lengths: /channel1.L50/channel2.L20/channel3.L100 (actually, specifying a length of last message is unnecessary, it's always a tail of POST data).

We can go even further with this idea. As I mentioned before, messages in our case differ only a little (it's a message id which is unique to each client, the rest (contents) of the message is the same for all recipients). We could save a lot of bandwidth (and some memory) by specifying a common part of messages only once. This can be achieved by implementing message suffixes. For example: push_stream_channels_path: /channel1.L2/channel2.L2/channel3.L2 POST data: aabbccMMMMMMM According to push_stream_channels_path there are 3 messages, each is 2 bytes long. And unused tail of POST data becomes a common suffix for all messages. Expected messages to be sent: aaMMMMMMM to channel1, bbMMMMMMM to channel2, ccMMMMMMM to channel3.

[wandenberg]

To not open multiple requests to your server when sending messages, you can take advantage of keepalive connections to publish messages faster. Your suggestion fits your use case but will make the code complex and probably will not works to others. Imagine if the differences between the messages are in the middle, or in the end, or a mix of them?! I think this is an application requirement and should be solved by the application.

[5lava]

We do use keep-alive (I mentioned persistent connections in my previous message).

I realize that suffix feature is pretty app-specific, perhaps we can patch the module by ourselves later. But I guess that ability to send messages in bulk by splitting POST data will fit many applications.

[wandenberg]

This was solved on tag 0.4.0 3d3a204177d3a7ab8a2858e04e792a6d11bf133f

[5lava]

push_stream_channels_path doesn't solve that.

[wandenberg]

Sorry, but yes, it fix the original issue "I often need to send the same message to multiple subscribers".

[5lava]

My bad, I should have started new issue regarding splitting POST data to multiple messages.