Pushing unbuffered binary data

[samsonradu]

Is it possible to push ffmpeg data into the /pub, so a continuous stream ?

For example, the below code throws a 413 because of the max_body_size limitation. Basically I'd need to disable buffering somehow and just let the data flow.

ffmpeg -i /dev/video0 -f mpeg1video http://localhost:88/pub?id=1

[ernaniaz]

Samson, look at NGiNX RTMP module, it does what you're looking for: https://github.com/arut/nginx-rtmp-module

[samsonradu]

@ernaniaz thanks, I've played with that before. Thing is I only want is to make long-running requests which keep posting data and nginx would forward the data to subscribers (through websockets).

Problem is my setup fails this check because the data flows subsequently:

https://github.com/wandenberg/nginx-push-stream-module/blob/master/src/ngx_http_push_stream_module_publisher.c#L250

Fe. in node it would look like this:

// HTTP Server to accept incomming MPEG Stream
var streamServer = require('http').createServer(function(request, response) {
  request.on('data', function(data){
    //do something with data;
  });
  request.on('end', function(){
    response.end();
  });
});

[samsonradu]

I think it's related to #99

http://nginx.org/en/docs/http/ngx_http_proxy_module.html?#proxy_request_buffering

[wandenberg]

@samsonradu the publisher location allow binary content but with defined content length. If you can cut the output of ffmpeg and post it to push stream module probably you will achieve your goal.

[samsonradu]

I was actually wondering if it's possible to just skip copying the body to a buffer here and just publish the request body directly. (might be a dumb idea)

https://github.com/wandenberg/nginx-push-stream-module/blob/master/src/ngx_http_push_stream_module_publisher.c#L271

So instead of this:

if (ngx_http_push_stream_add_msg_to_channel(mcf, r->connection->log, requested_channel->channel, buf->pos, ngx_buf_size(buf), event_id, event_type, cf->store_messages, r->pool) != NGX_OK) {
    //...
} 

Do something like this:

chain = r->request_body->bufs;
while ((chain != NULL) && (chain->buf != NULL)) {
    if (ngx_http_push_stream_add_msg_to_channel(mcf, r->connection->log, requested_channel->channel, chain->buf->pos, ngx_buf_size(chain->buf), event_id, event_type, cf->store_messages, r->pool) != NGX_OK) {
      //...
  } 

  chain = chain->next;
}

[wandenberg]

It isn't that simple. You are not dealing with a big body divided in multiple buffers. This would be a stream body. If you do a loop like that the worker will be blocked, but of course you can try to change the module to achieve your goals.