Erasing read messages

[conectagames]

Hello Is it possible for the stored messages to be automatically erased after having been read ? Thxs

[wandenberg]

short answer: No The main target to stored messages is to deliver the message to a subscriber that was offline when the message was published. How many subscribers were offline? How many we will wait for be online again before discard the message? These are questions that make difficult to construct a generic software. So the policy to discard stored messages are based only on the time to live. (may be in the future we use the "memory full" as another policy)

Do you have a specific problem that the message ttl does not fit?

[conectagames]

Hello. I am thinking in a scenario where each subscriber has its own channel, and when it consumes the messages the messages are gone; but stored or kept when the subscriber has not yet subscribed or consumed the messages. Is this possible ? Now messages are lost if the subscriber is not yet listening.

[wandenberg]

No. You can set to store the messages for a "big time", and limit or not the queue size. Be aware that you may need to increase the shared memory to keep messages for a longer period.

[conectagames]

I understand, but memory is being used by messages that have already been read. Perhaps it could be added in the future. Great module!

[msurguy]

@conectagames perhaps some database like Redis or MySQL would be a better fit for storing the messages and erasing them when they have been read? In my opinion this module should only care about broadcasting messages to many clients instead of taking on functions of a database.

[marul2]

I work in the same scenario of conectagames. Each subscriber has its own channel. An easy and reliable way to message comunication between server and external clients (game apps, mobile phones).

@msurguy , the problem is not mysql, redis, etc.. but the external comunication. nginx-push-stream-module is great handling websockets, sse, long-polling, etc...

I would like to make two requests for future:

Option to automatically delete the message when read. if no consumer read channel with a timeout, the channel is automatically deleted and module rejects post a new message.

Thank you and sorry for my english.

[wandenberg]

@marul2, as I said before automatically delete the message when read is not a generic solution even when working on an environment where each subscriber has its own channel. Imagine if the same subscriber open two connections to your server, like two browser tabs, the first will get the message and the second don't. Is hard to find a solution that will fit to everyone.

Concerning your feature request it already is implemented, at least part of it. When a channel with no messages or subscribers reach an inactivity time it is deleted. Reject a post to a channel that was deleted needs to keep a status of that channel only for that. If not, will be impossible to distinguish between a deleted channel and a channel that never existed. Probably will be easier implement this solution at your application, since it is not generic enough to be part of the core of the module. Despite of that, suggestions, contributions and feature requests are welcome.

[marul2]

You are right that the push/pull paradigm prohibits delete the message when read and it is not generic enough to be part of the core of the module.

However for my personal use:

in ngx_http_push_stream_module_utils At the end of ngx_http_push_stream_send_response_message function:

msg->expires = msg->time;

It seems to work. is it correct?

Thant you for so great module!

[wandenberg]

@marul2 doing this change will make the message be removed from the queue on the next cleanup cycle. But it still on the list until there. This routine runs at each 5 seconds. When this happens, the expired messages are moved to the trash queue and stay there for more 10 seconds at least. Then they are released. So it works, respecting the times explained before.