Bandwidth usage

[cbarburescu]

Hello,

I am testing this solution with a test video (1080p, 30 minutes long, bitrate 1700 kb/s), using HLS.

I am using artillery for testing, with max 10 simultaneous users and 1 concurrent downloaded segment/ user.

I am using 2 vms from google cloud, region europe-west4-a (Netherlands) - one for streaming and one for testing. I have limited the nic of the streaming vm at 200mbps (to replicate the real setup I will have - a baremetal server with a 200mbps connection).

I am monitoring the egress of the streaming vm, which constantly sits at 180 mbps:

My question is: what is consuming so much bandhwidth and how can I optimize it, in order to support more users on that connection? If I have a video with a bitrate of 2mb/s, then 10 users should generate an egress of 20mbps (not considering the metadata). I am puzzled why it is taking 9 times more (180 = 9 * 20 mbps).

Furthermore, if I load the video from a browser (location: Denmark)§ at the same time that I am doing the test, the loading time of the first segment is as longs as 5-6 seconds, compared to <1 second (when the test script does not run).

Nginx config file:

user www-data;
worker_processes 10;
worker_rlimit_nofile 8192;
pid /run/nginx.pid;

events {
    worker_connections 4096;
}

http {
    server {
        listen 443 default_server;
        # listen 443 ssl default_server;
    # include snippets/self-signed.conf;
        # include snippets/ssl-params.conf;

        server_name streaming.example.dk;   

    root /var/www/streaming.example.dk;

        location / {
            index  index.html index.htm;
        }
    }

    server {
        listen 80;

        # vod mode
        vod_mode mapped;

        # https
        # vod_segments_base_url http://streaming.example.dk;
        # vod_base_url https://streaming.example.dk;

        # vod caches
        vod_metadata_cache metadata_cache 3072;
        vod_response_cache response_cache 64m;
        vod_mapping_cache mapping_cache 64m;

        # gzip manifests
        gzip on;
        gzip_types application/vnd.apple.mpegurl;

        # file handle caching
        open_file_cache          max=10000 inactive=5m;
        open_file_cache_valid    2m;
        open_file_cache_min_uses 1;
        open_file_cache_errors   on;
        aio on;

        # vod status page
        location /vod_status {
            vod_status;
            access_log off;
        }

        location ^~ /video/ {
            alias /etc/nginx/json/;
            vod hls;

            add_header Access-Control-Allow-Headers '*';
            add_header Access-Control-Expose-Headers 'Server,range,Content-Length,Content-Range';
            add_header Access-Control-Allow-Methods 'GET, HEAD, OPTIONS';
            add_header Access-Control-Allow-Origin '*';
            expires 100d;
        }
    }
}

[cbarburescu]

I figured it out: the high bandhwidth usage comes from the segment size, which is 10s by default (and thus, 10 times bigger than the size per second).

The solution I found is to set the vod segments to the size of the gop, using the following configuration:

        vod_segment_duration 2000;
        vod_align_segments_to_key_frames on;
        vod_manifest_segment_durations_mode accurate;

This can potentially reduce the size of a segment by 3 to 5 times, depending on your gop size (between 2 and 5 seconds).

[francoism90]

@cbarburescu Interesting stats!

To further reduce bandwidth, you could use proxies, like caching segments, etc.