Not receiving any streams on a subscribed socket

[blazczak]

Kudos @wandenberg for your work on the module.

I've written a C-based socket client which interfaces with nginx/nginx-push-stream-module and is meant to receive streams from the server after they were published to one of the channels, and to send data to the server for other subscribers to consume - all on the same non-blocking socket connection.

The nginx module configuration boils down to the following:

server { 
    listen 1234; 

    location /channels-stats {
        # activate channels statistics mode for this location
        push_stream_channels_statistics;

        # query string based channel path
        push_stream_channels_path                    $arg_id;
    }

    location /pub {
        # activate publisher (admin) mode for this location
        push_stream_publisher admin;

        # query string based channel path
        push_stream_channels_path                    $arg_id;

        push_stream_channel_info_on_publish          off;

        # store messages in memory
        push_stream_store_messages                   on;
    }

    location ~ /sub/(.*) {
        # activate streaming mode for this location
        push_stream_subscriber;

        # positional channel path
        push_stream_channels_path                     $1;

        push_stream_last_received_message_time        "$arg_time";
    }

    location / { 
        root /var/www/html;
    } 

}

push_stream_channel_info_on_publish is off so that I receive no extraneous JSON response.

After the initial non-blocking socket connection, I send an HTTP GET (connect info redacted):

GET /sub/channel HTTP/1.1
> Host: xxxxxxxxxxx
> User-Agent: curl/7.54.0
> Accept: */*
(newline)

and I would expect to receive the stream of data when it's posted to the channel. The POST writes from the client on the same (non-blocking) socket are taking place, because I can see the data streaming in continuously after issuing a command-line curl to subscribe to the channel curl -s -v --no-buffer 'http://myhost:myport/sub/channel'.

The POSTs to the server by the client produce 200 OK responses, encoded "chunked", which are seen and read by the socket client.

xxx.xxx.xxx.xxx - - [06/Dec/2020:22:26:14 +0000] "POST /pub?id=channel HTTP/1.1" 200 0 "-" "curl/7.54.1"

I can see the connection time out periodically, as if nothing was sent in response to the GET, and the C client is designed to reestablish the connection and reissues another GET.

So, essentially the only data being read on the socket are the responses from the server to my POSTs, but no data from the subscribed channel.

I feel like there's something minor that I'm missing. Is it a configuration or a timing issue? Do I need something additional in the socket configuration? Are the subscribed data reads trumpled by the POST writes and then the 200 responses? Let me know what you think as to what could be causing this.

[blazczak]

Could not get the subscription GET and publish POSTs to play nice over one socket, and it wasn't a timing issue - I ended up using two sockets for this - one for subscribing to and receiving streamed data, the other for publishing data to the server for dissemination and consuming the incoming 200 OKs.

If someone has a working setup/configuration for consuming streams from+publishing to push-stream-module over one socket, let me know.

[wandenberg]

@blazczak sorry for the delay. As you find out, the module does not support to write/read in one socket for the "normal" subscriber configuration. One easier way of doing it would be using a C-client for WebSocket (or doing one). When you set the subscriber to be websocket mode it allows you to publish messages using the same socket (of course implementing the WebSocket protocol). This limitation on the "normal" mode is connected with the usage the module was initially built for that is to send/receive messages directly to browsers using pure HTML + JS. (It might be reviewed in the future)