search

SRGSSR / nginx-vod-module

NGINX-based MP4 Repackager
GNU Affero General Public License v3.0
2 stars 1 forks source link

readme

NGINX-based VOD Packager

nginx-vod-module Build Status

Join the list of organizations using this video packager project.

For live video streaming, please use Media-Framework.

Features

Limitations

Compilation

Dependencies

In general, if you have the dependencies that are required to build nginx, you should be able to build nginx-vod-module. However, some optional features of this module depend on additional packages. The module detects these packages during configure - if a package is missing, the respective feature will be disabled.

The optional features are:

  1. Thumbnail capture & volume map - depend on ffmpeg (3.0 or newer)
  2. Audio filtering (for changing playback rate / gain) - depends on ffmpeg (3.0 or newer) and also on libfdk_aac. Due to licensing issues, libfdk_aac is not built into kaltura ffmpeg packages
  3. Encryption / decryption (DRM / HLS AES) - depends on openssl
  4. DFXP captions - depends on libxml2
  5. UTF-16 encoded SRT files - depends on iconv

Build

To link statically against nginx, cd to nginx source directory and execute:

./configure --add-module=/path/to/nginx-vod-module
make
make install

To compile as a dynamic module (nginx 1.9.11+), use:

./configure --add-dynamic-module=/path/to/nginx-vod-module

In this case, the load_module directive should be used in nginx.conf in order to load the module.

Optional recommended settings:

  1. --with-file-aio - enable asynchronous I/O support, highly recommended, relevant only to local and mapped modes
  2. --with-threads (nginx 1.7.11+) - enable asynchronous file open using thread pool (also requires vod_open_file_thread_pool in nginx.conf), relevant only to local and mapped modes
  3. --with-cc-opt="-O3 -mpopcnt" - enable additional compiler optimizations (we saw about 8% reduction in the mp4 parse time and frame processing time compared to the nginx default -O)

Debug settings:

  1. --with-debug - enable debug messages (also requires passing debug in the error_log directive in nginx.conf).
  2. --with-cc-opt="-O0" - disable compiler optimizations (for debugging with gdb)

C Macro Configurations:

  1. --with-cc-opt="-DNGX_VOD_MAX_TRACK_COUNT=256 -mavx2" - increase the maximum track count (preferably to multiples of 64). It's recommended to enable vector extensions (AVX2) as well.

Installation

RHEL/CentOS 6/7 RPM

# rpm -ihv http://installrepo.kaltura.org/releases/kaltura-release.noarch.rpm
# yum install kaltura-nginx

Debian/Ubuntu deb package

Ubuntu NOTE: before trying to install kaltura-nginx, you must also make sure the multiverse repo is enabled

For Debian Wheezy [7], Debian Jessie [8], Ubuntu 14.04 and 14.10, add this repo:

# wget -O - http://installrepo.kaltura.org/repo/apt/debian/kaltura-deb-curr.gpg.key|apt-key add -
# echo "deb [arch=amd64] http://installrepo.kaltura.org/repo/apt/debian propus main" > /etc/apt/sources.list.d/kaltura.list

For Ubuntu 16.04, 16.10 add this repo:

# wget -O - http://installrepo.kaltura.org/repo/apt/xenial/kaltura-deb-curr-256.gpg.key|apt-key add -
# echo "deb [arch=amd64] http://installrepo.kaltura.org/repo/apt/xenial propus main" > /etc/apt/sources.list.d/kaltura.list

For Ubuntu 20.04 add this repo:

# wget -O - http://installrepo.kaltura.org/repo/aptn/focal/kaltura-deb-256.gpg.key|apt-key add -
# echo "deb [arch=amd64] http://installrepo.kaltura.org/repo/aptn/focal quasar main" > /etc/apt/sources.list.d/kaltura.list

Then install the kaltura-nginx package:

# apt-get update
# apt-get install kaltura-nginx

If you wish to make use of the following features:

You will also need to install the kaltura-ffmpeg (>= 3.1) package.

URL structure

Basic URL structure

The basic structure of an nginx-vod-module URL is: http://<domain>/<location>/<fileuri>/<filename>

Where:

Multi URL structure

Multi URLs are used to encode several URLs on a single URL. A multi URL can be used to specify the URLs of several different MP4 files that should be included together in a DASH MPD for example.

The structure of a multi URL is: http://<domain>/<location>/<prefix>,<middle1>,<middle2>,<middle3>,<postfix>.urlset/<filename>

The sample URL above represents 3 URLs:

The suffix .urlset (can be changed using vod_multi_uri_suffix) indicates that the URL should be treated as a multi URL. For example - the URL http://example.com/hls/videos/big_buck_bunny_,6,9,15,00k.mp4.urlset/master.m3u8 will return a manifest containing:

URL path parameters

The following parameters are supported on the URL path:

Filename structure

The structure of filename is: <basename>[<seqparams>][<fileparams>][<trackparams>][<langparams>].<extension>

Where:

Mapping response format

When configured to run in mapped mode, nginx-vod-module issues an HTTP request to a configured upstream server in order to receive the layout of media streams it should generate. The response has to be in JSON format.

This section contains a few simple examples followed by a reference of the supported objects and fields. But first, a couple of definitions:

  1. Source Clip - a set of audio and/or video frames (tracks) extracted from a single media file
  2. Generator - a component that can generate audio/video frames. Currently, the only supported generator is the silence generator.
  3. Filter - a manipulation that can be applied on audio/video frames. The following filters are supported:
    • rate (speed) change - applies to both audio and video
    • audio volume change
    • mix - can be used to merge several audio tracks together, or to merge the audio of source A with the video of source B
  4. Clip - the result of applying zero or more filters on a set of source clips
  5. Dynamic Clip - a clip whose contents is not known in advance, e.g. targeted ad content
  6. Sequence - a set of clips that should be played one after the other.
  7. Set - several sequences that play together as an adaptive set, each sequence must have the same number of clips.

Simple mapping

The JSON below maps the request URI to a single MP4 file:

{
    "sequences": [
        {
            "clips": [
                {
                    "type": "source",
                    "path": "/path/to/video.mp4"
                }
            ]
        }
    ]
}

When using multi URLs, this is the only allowed JSON pattern. In other words, it is not possible to combine more complex JSONs using multi URL.

Adaptive set

As an alternative to using multi URL, an adaptive set can be defined via JSON:

{
    "sequences": [
        {
            "clips": [
                {
                    "type": "source",
                    "path": "/path/to/bitrate1.mp4"
                }
            ]
        },
        {
            "clips": [
                {
                    "type": "source",
                    "path": "/path/to/bitrate2.mp4"
                }
            ]
        }
    ]
}

Playlist

The JSON below will play 35 seconds of video1 followed by 22 seconds of video2:

{
    "durations": [ 35000, 22000 ],
    "sequences": [
        {
            "clips": [
                {
                    "type": "source",
                    "path": "/path/to/video1.mp4"
                },
                {
                    "type": "source",
                    "path": "/path/to/video2.mp4"
                }
            ]
        }
    ]
}

Filters

The JSON below takes video1, plays it at x1.5 and mixes the audio of the result with the audio of video2, after reducing it to 50% volume:

{
    "sequences": [
        {
            "clips": [
                {
                    "type": "mixFilter",
                    "sources": [
                        {
                            "type": "rateFilter",
                            "rate": 1.5,
                            "source": {
                                "type": "source",
                                "path": "/path/to/video1.mp4"
                            }
                        },
                        {
                            "type": "gainFilter",
                            "gain": 0.5,
                            "source": {
                                "type": "source",
                                "path": "/path/to/video2.mp4",
                                "tracks": "a1"
                            }
                        }
                    ]
                }
            ]
        }
    ]
}

Continuous live

The JSON below is a sample of a continuous live stream (=a live stream in which all videos have exactly the same encoding parameters). In practice, this JSON will have to be generated by some script, since it is time dependent. (see test/playlist.php for a sample implementation)

{
    "playlistType": "live",
    "discontinuity": false,
    "segmentBaseTime": 1451904060000,
    "firstClipTime": 1451917506000,
    "durations": [83000, 83000],
    "sequences": [
        {
            "clips": [
                {
                    "type": "source",
                    "path": "/path/to/video1.mp4"
                },
                {
                    "type": "source",
                    "path": "/path/to/video2.mp4"
                }
            ]
        }
    ]
}

Non-continuous live

The JSON below is a sample of a non-continuous live stream (=a live stream in which the videos have different encoding parameters). In practice, this JSON will have to be generated by some script, since it is time dependent (see test/playlist.php for a sample implementation)

{
    "playlistType": "live",
    "discontinuity": true,
    "initialClipIndex": 171,
    "initialSegmentIndex": 153,
    "firstClipTime": 1451918170000,
    "durations": [83000, 83000],
    "sequences": [
        {
            "clips": [
                {
                    "type": "source",
                    "path": "/path/to/video1.mp4"
                },
                {
                    "type": "source",
                    "path": "/path/to/video2.mp4"
                }
            ]
        }
    ]
}

Mapping reference

Set (top level object in the mapping JSON)

Mandatory fields:

Optional fields:

Live fields:

Sequence

Mandatory fields:

Optional fields:

Clip (abstract)

Mandatory fields:

Optional fields:

Source clip

Mandatory fields:

Optional fields:

Rate filter clip

Mandatory fields:

Gain filter clip

Mandatory fields:

Mix filter clip

Mandatory fields:

Concat clip

Mandatory fields:

Optional fields:

Dynamic clip

Mandatory fields:

Notification

Mandatory fields:

Closed Captions

Mandatory fields:

Optional fields:

Security

Authorization

CDN-based delivery

Media packaged by nginx-vod-module can be protected using CDN tokens, this works as follows:

In this setup it also highly recommended to block direct access to the origin server by authenticating the CDN requests. Without this protection, a user who somehow gets the address of the origin will be able to bypass the CDN token enforcement. If using Akamai, this can be accomplished using https://github.com/refractalize/nginx_mod_akamai_g2o. For other CDNs, it may be possible to configure the CDN to send a secret header to the origin and then simply enforce the header using an nginx if statement:

        if ($http_x_secret_origin_header != "secret value") {
            return 403;
        }

In addition to the above, most CDNs support other access control settings, such as geo-location. These restrictions are completely transparent to the origin and should work well.

Direct delivery

Deployments in which the media is pulled directly from nginx-vod-module can protect the media using nginx access control directives, such allow, deny, or access_by_lua (for more complex scenarios).

In addition, it is possible to build a token based solution (as detailed in the previous section) without a CDN, by having the nginx server validate the token. The module https://github.com/kaltura/nginx-akamai-token-validate-module can be used to validate Akamai tokens. Locations on which the module is enabled will return 403 unless the request contains a valid Akamai token. See the readme of this module for more details.

URL encryption

As an alternative to tokenization, URL encryption can be used to prevent an attacker from being able to craft a playable URL. URL encryption can be implemented with https://github.com/kaltura/nginx-secure-token-module, and is supported for HLS and DASH (with manifest format set to segmentlist).

In terms of security, the main advantage of CDN tokens over URL encryption is that CDN tokens usually expire, while encrypted URLs do not (someone who obtains a playable URL will be able to use it indefinitely)

Media encryption

Nginx-vod-module supports AES-128 and SAMPLE-AES HLS encryption schemes. The main difference between media encryption and DRM (detailed below) is the mechanism used to transfer the encryption key to the client. With media encryption the key is fetched by the client by performing a simple GET request to nginx-vod-module, while with DRM the key is returned inside a vendor specific license response.

Media encryption reduces the problem of securing the media to the need to secure the encryption key. The media segment URLs (which compose the vast majority of the traffic) can be completely unprotected, and easily cacheable by any proxies between the client and servers (unlike tokenization). The encryption key request can then be protected using one of the methods mentioned above (CDN tokens, nginx access rules etc.).

In addition, it is possible to configure nginx-vod-module to return the encryption key over HTTPS while having the segments delivered over HTTP. The way to configure this is to set vod_segments_base_url to http://nginx-vod-host and set vod_base_url to https://nginx-vod-host.

DRM

Nginx-vod-module has the ability to perform on-the-fly encryption for MPEG DASH (CENC), MSS Play Ready and FairPlay HLS. As in the case of media encryption, the encryption is performed while serving a video/audio segment to the client, therefore, when working with DRM it is recommended not to serve the content directly from nginx-vod-module to end-users. A more scalable architecture would be to use proxy servers or a CDN in order to cache the encrypted segments.

In order to perform the encryption, nginx-vod-module needs several parameters, including key & key_id, these parameters are fetched from an external server via HTTP GET requests. The vod_drm_upstream_location parameter specifies an nginx location that is used to access the DRM server, and the request uri is configured using vod_drm_request_uri (this parameter can include nginx variables). The response of the DRM server is a JSON, with the following format:

[{
    "pssh": [{
            "data": "CAESEGMyZjg2MTczN2NjNGYzODIaB2thbHR1cmEiCjBfbmptaWlwbXAqBVNEX0hE", 
            "uuid": "edef8ba9-79d6-4ace-a3c8-27dcd51d21ed"
        }], 
    "key": "GzoNU9Dfwc//Iq3/zbzMUw==", 
    "key_id": "YzJmODYxNzM3Y2M0ZjM4Mg=="
}]
Sample configurations

Apple FairPlay HLS:

location ~ ^/fpshls/p/\d+/(sp/\d+/)?serveFlavor/entryId/([^/]+)/(.*) {
    vod hls;
    vod_hls_encryption_method sample-aes;
    vod_hls_encryption_key_uri "skd://entry-$2";
    vod_hls_encryption_key_format "com.apple.streamingkeydelivery";
    vod_hls_encryption_key_format_versions "1";

    vod_drm_enabled on;
    vod_drm_request_uri "/udrm/system/ovp/$vod_suburi";

    vod_last_modified_types *;
    add_header Access-Control-Allow-Headers '*';
    add_header Access-Control-Expose-Headers 'Server,range,Content-Length,Content-Range';
    add_header Access-Control-Allow-Methods 'GET, HEAD, OPTIONS';
    add_header Access-Control-Allow-Origin '*';
    expires 100d;
}

Common Encryption HLS:

location ~ ^/cenchls/p/\d+/(sp/\d+/)?serveFlavor/entryId/([^/]+)/(.*) {
    vod hls;
    vod_hls_encryption_method sample-aes-cenc;
    vod_hls_encryption_key_format "urn:uuid:edef8ba9-79d6-4ace-a3c8-27dcd51d21ed";
    vod_hls_encryption_key_format_versions "1";

    vod_drm_enabled on;
    vod_drm_request_uri "/udrm/system/ovp/$vod_suburi";

    vod_last_modified_types *;
    add_header Access-Control-Allow-Headers '*';
    add_header Access-Control-Expose-Headers 'Server,range,Content-Length,Content-Range';
    add_header Access-Control-Allow-Methods 'GET, HEAD, OPTIONS';
    add_header Access-Control-Allow-Origin '*';
    expires 100d;
}
Verified configurations

Following is a list of configurations that were tested and found working:

Performance recommendations

  1. For medium/large scale deployments, don't have users play the videos directly from nginx-vod-module. Since all the different streaming protocols supported by nginx vod are HTTP based, they can be cached by standard HTTP proxies / CDNs. For medium scale add a layer of caching proxies between the vod module and the end users (can use standard nginx servers with proxy_pass & proxy_cache). For large scale deployments, it is recommended to use a CDN (such as Akamai, Level3 etc.).

    In general, it's best to have nginx vod as close as possible to where the mp4 files are stored, and have the caching proxies as close as possible to the end users.

  2. Enable nginx-vod-module caches:

    • vod_metadata_cache - saves the need to re-read the video metadata for each segment. This cache should be rather large, in the order of GBs.
    • vod_response_cache - saves the responses of manifest requests. This cache may not be required when using a second layer of caching servers before nginx vod. No need to allocate a large buffer for this cache, 128M is probably more than enough for most deployments.
    • vod_mapping_cache - for mapped mode only, few MBs is usually enough.
    • nginx's open_file_cache - caches open file handles.

    The hit/miss ratios of these caches can be tracked by enabling performance counters (vod_performance_counters) and setting up a status page for nginx vod (vod_status)

  3. In local & mapped modes, enable aio. - nginx has to be compiled with aio support, and it has to be enabled in nginx conf (aio on). You can verify it works by looking at the performance counters on the vod status page - read_file (aio off) vs. async_read_file (aio on)
  4. In local & mapped modes, enable asynchronous file open - nginx has to be compiled with threads support, and vod_open_file_thread_pool has to be specified in nginx.conf. You can verify it works by looking at the performance counters on the vod status page - open_file vs. async_open_file. Note that open_file may be nonzero with vod_open_file_thread_pool enabled, due to the open file cache - open requests that are served from cache will be counted as synchronous open_file.
  5. When using DRM enabled DASH/MSS, if the video files have a single nalu per frame, set vod_min_single_nalu_per_frame_segment to non-zero.
  6. The muxing overhead of the streams generated by this module can be reduced by changing the following parameters:
    • HDS - set vod_hds_generate_moof_atom to off
    • HLS - set vod_hls_mpegts_align_frames to off and vod_hls_mpegts_interleave_frames to on

  7. Enable gzip compression on manifest responses -

    gzip_types application/vnd.apple.mpegurl video/f4m application/dash+xml text/xml

  8. Apply common nginx performance best practices, such as tcp_nodelay=on, client_header_timeout etc.

Configuration directives - base

vod

Enables the nginx-vod module on the enclosing location. The allowed values for segmenter are:

  1. none - serves the MP4 files as is / clipped
  2. dash - Dynamic Adaptive Streaming over HTTP packager
  3. hds - Adobe HTTP Dynamic Streaming packager
  4. hls - Apple HTTP Live Streaming packager
  5. mss - Microsoft Smooth Streaming packager
  6. thumb - thumbnail capture
  7. volume_map - audio volume map

vod_mode

Sets the file access mode - local, remote or mapped (see the features section above for more details)

vod_status

Enables the nginx-vod status page on the enclosing location. The following query params are supported:

Configuration directives - segmentation

vod_segment_duration

Sets the segment duration in milliseconds. It is highly recommended to use a segment duration that is a multiple of the GOP duration. If the segment duration is not a multiple of GOP duration, and vod_align_segments_to_key_frames is enabled, there could be significant differences between the segment duration that is reported in the manifest and the actual segment duration. This could also lead to the appearance of empty segments within the stream.

vod_live_window_duration

Sets the total duration in milliseconds of the segments that should be returned in a live manifest. If the value is positive, nginx vod returns a range of maximum vod_live_window_duration milliseconds, ending at the current server time. If the value is negative, nginx vod returns a range of maximum -vod_live_window_duration milliseconds from the end of the mapping json. If the value is set to zero, the live manifest will contain all the segments that are fully contained in the mapping json time frame.

vod_force_playlist_type_vod

Generate a vod stream even when the media set has playlistType=live. Enabling this setting has the following effects:

  1. Frame timestamps will be continuous and start from zero
  2. Segment indexes will start from one
  3. In case of HLS, the returned manifest will have both #EXT-X-PLAYLIST-TYPE:VOD and #EXT-X-ENDLIST

This can be useful for clipping vod sections out of a live stream.

vod_force_continuous_timestamps

Generate continuous timestamps even when the media set has gaps (gaps can created by the use of clipTimes) If ID3 timestamps are enabled (vod_hls_mpegts_output_id3_timestamps), they contain the original timestamps that were set in clipTimes.

vod_bootstrap_segment_durations

Adds a bootstrap segment duration in milliseconds. This setting can be used to make the first few segments shorter than the default segment duration, thus making the adaptive bitrate selection kick-in earlier without the overhead of short segments throughout the video.

vod_align_segments_to_key_frames

When enabled, the module forces all segments to start with a key frame. Enabling this setting can lead to differences between the actual segment durations and the durations reported in the manifest (unless vod_manifest_segment_durations_mode is set to accurate).

vod_segment_count_policy

Configures the policy for calculating the segment count, for segment_duration = 10 seconds:

vod_manifest_duration_policy

Configures the policy for calculating the duration of a manifest containing multiple streams:

vod_manifest_segment_durations_mode

Configures the calculation mode of segment durations within manifest requests:

vod_media_set_override_json

This parameter provides a way to override portions of the media set JSON (mapped mode only). For example, vod_media_set_override_json '{"clipTo":20000}' clips the media set to 20 sec. The parameter value can contain variables.

Configuration directives - upstream

vod_upstream_location

Sets an nginx location that is used to read the MP4 file (remote mode) or mapping the request URI (mapped mode).

vod_remote_upstream_location

Sets an nginx location that is used to read the MP4 file on remote or mapped mode. If this directive is set on mapped mode, the module reads the MP4 files over HTTP, treating the paths in the mapping JSON as URIs (the default behavior is to read from local files)

vod_max_upstream_headers_size

Sets the size that is allocated for holding the response headers when issuing upstream requests (to vod_xxx_upstream_location).

vod_upstream_extra_args

Extra query string arguments that should be added to the upstream request (remote/mapped modes only). The parameter value can contain variables.

vod_media_set_map_uri

Sets the uri of media set mapping requests, the parameter value can contain variables. In case of multi url, $vod_suburi will be the current sub uri (a separate request is issued per sub URL)

vod_path_response_prefix

Sets the prefix that is expected in URI mapping responses (mapped mode only).

vod_path_response_postfix

Sets the postfix that is expected in URI mapping responses (mapped mode only).

vod_max_mapping_response_size

Sets the maximum length of a path returned from upstream (mapped mode only).

Configuration directives - fallback

vod_fallback_upstream_location

Sets an nginx location to which the request is forwarded after encountering a file not found error (local/mapped modes only).

vod_proxy_header_name

Sets the name of an HTTP header that is used to prevent fallback proxy loops (local/mapped modes only).

vod_proxy_header_value

Sets the value of an HTTP header that is used to prevent fallback proxy loops (local/mapped modes only).

Configuration directives - performance

vod_metadata_cache

Configures the size and shared memory object name of the video metadata cache. For MP4 files, this cache holds the moov atom.

vod_mapping_cache

Configures the size and shared memory object name of the mapping cache for vod (mapped mode only).

vod_live_mapping_cache

Configures the size and shared memory object name of the mapping cache for live (mapped mode only).

vod_response_cache

Configures the size and shared memory object name of the response cache. The response cache holds manifests and other non-video content (like DASH init segment, HLS encryption key etc.). Video segments are not cached.

vod_live_response_cache

Configures the size and shared memory object name of the response cache for time changing live responses. This cache holds the following types of responses for live: DASH MPD, HLS index M3U8, HDS bootstrap, MSS manifest.

vod_initial_read_size

Sets the size of the initial read operation of the MP4 file.

vod_max_metadata_size

Sets the maximum supported video metadata size (for MP4 - moov atom size)

vod_max_frames_size

Sets the limit on the total size of the frames of a single segment

vod_max_frame_count

Sets the limit on the total count of the frames read to serve non segment (e.g. playlist) request.

vod_segment_max_frame_count

Sets the limit on the total count of the frames read to serve segment request.

vod_cache_buffer_size

Sets the size of the cache buffers used when reading MP4 frames.

vod_open_file_thread_pool

Enables the use of asynchronous file open via thread pool. The thread pool must be defined with a thread_pool directive, if no pool name is specified the default pool is used. This directive is supported only on nginx 1.7.11 or newer when compiling with --add-threads. Note: this directive currently disables the use of nginx's open_file_cache by nginx-vod-module

vod_output_buffer_pool

Pre-allocates buffers for generating response data, saving the need allocate/free the buffers on every request.

vod_performance_counters

Configures the shared memory object name of the performance counters

Configuration directives - url structure

vod_base_url

Sets the base URL (scheme + domain) that should be returned in manifest responses. The parameter value can contain variables, if the parameter evaluates to an empty string, relative URLs will be used. If the parameter evaluates to a string ending with /, it is assumed to be a full URL - the module only appends the file name to it, instead of a full URI. If not set, the base URL is determined as follows:

  1. If the request did not contain a host header (HTTP/1.0) relative URLs will be returned
  2. Otherwise, the base URL will be $scheme://$http_host The setting currently affects only HLS and DASH. In MSS and HDS, relative URLs are always returned.

vod_segments_base_url

Sets the base URL (scheme + domain) that should be used for delivering video segments. The parameter value can contain variables, if the parameter evaluates to an empty string, relative URLs will be used. If not set, vod_base_url will be used. The setting currently affects only HLS.

vod_multi_uri_suffix

A URL suffix that is used to identify multi URLs. A multi URL is a way to encode several different URLs that should be played together as an adaptive streaming set, under a single URL. When the default suffix is used, an HLS set URL may look like: http://host/hls/common-prefix,bitrate1,bitrate2,common-suffix.urlset/master.m3u8

vod_clip_to_param_name

The name of the clip to request parameter.

vod_clip_from_param_name

The name of the clip from request parameter.

vod_tracks_param_name

The name of the tracks request parameter.

vod_time_shift_param_name

The name of the shift request parameter.

vod_speed_param_name

The name of the speed request parameter.

vod_lang_param_name

The name of the language request parameter.

vod_force_sequence_index

Use sequence index in segment uris even if there is only one sequence

Configuration directives - response headers

vod_expires

Sets the value of the "Expires" and "Cache-Control" response headers for successful requests. This directive is similar to nginx's built-in expires directive, except that it only supports the expiration interval scenario (epoch, max, off, day time are not supported) Main motivation for using this directive instead of the built-in expires is to have different expiration for VOD and dynamic live content. If this directive is not specified, nginx-vod-module will not set the "Expires" / "Cache-Control" headers. This setting affects all types of requests in VOD playlists and segment requests in live playlists.

vod_expires_live

Same as vod_expires (above) for live requests that are not time dependent and not segments (e.g. HLS - master.m3u8, HDS - manifest.f4m).

vod_expires_live_time_dependent

Same as vod_expires (above) for live requests that are time dependent (HLS - index.m3u8, HDS - bootstrap.abst, MSS - manifest, DASH - manifest.mpd).

vod_last_modified

Sets the value of the Last-Modified header returned on the response, by default the module does not return a Last-Modified header. The reason for having this parameter here is in order to support If-Modified-Since / If-Unmodified-Since. Since nginx's builtin ngx_http_not_modified_filter_module runs before any other header filter module, it will not see any headers set by add_headers / more_set_headers. This makes nginx always reply as if the content changed (412 for If-Unmodified-Since / 200 for If-Modified-Since) For live requests that are not segments (e.g. live DASH MPD), Last-Modified is set to the current server time.

vod_last_modified_types

Sets the MIME types for which the Last-Modified header should be set. The special value "*" matches any MIME type.

Configuration directives - ad stitching (mapped mode only)

vod_dynamic_mapping_cache

Configures the size and shared memory object name of the cache that stores the mapping of dynamic clips.

vod_dynamic_clip_map_uri

Sets the uri that should be used to map dynamic clips. The parameter value can contain variables, specifically, $vod_clip_id contains the id of the clip that should be mapped. The expected response from this uri is a JSON containing a concat clip object.

vod_source_clip_map_uri

Sets the uri that should be used to map source clips defined using the clipIds property of concat. The parameter value can contain variables, specifically, $vod_clip_id contains the id of the clip that should be mapped. The expected response from this uri is a JSON containing a source clip object.

vod_redirect_segments_url

Sets a url to which requests for segments should be redirected. The parameter value can contain variables, specifically, $vod_dynamic_mapping contains a serialized representation of the mapping of dynamic clips.

vod_apply_dynamic_mapping

Maps dynamic clips to concat clips using the given expression, previously generated by $vod_dynamic_mapping. The parameter value can contain variables.

vod_notification_uri

Sets the uri that should be used to issue notifications. The parameter value can contain variables, specifically, $vod_notification_id contains the id of the notification that is being fired. The response from this uri is ignored.

Configuration directives - DRM / encryption

vod_secret_key

Sets the seed that is used to generate the TS encryption key and DASH/MSS encryption IVs. The parameter value can contain variables, and will usually have the structure "secret-$vod_filepath". See the list of nginx variables added by this module below.

vod_encryption_iv_seed

Sets the seed that is used to generate the encryption IV, currently applies only to HLS/fMP4 with AES-128 encryption. The parameter value can contain variables.

vod_drm_enabled

When enabled, the module encrypts the media segments according to the response it gets from the drm upstream. Currently supported only for dash and mss (play ready).

vod_drm_single_key

When enabled, the module requests the drm info only for the first sequence and applies it to all sequences. When disabled, the drm info is requested for each sequence separately. In addition, in DASH, enabling this setting makes the module place the ContentProtection tag under AdaptationSet, otherwise, it is placed under Representation.

vod_drm_clear_lead_segment_count

Sets the number of clear (unencrypted) segments in the beginning of the stream. A clear lead enables the player to start playing without having to wait for the license response.

vod_drm_max_info_length

Sets the maximum length of a drm info returned from upstream.

vod_drm_upstream_location

Sets the nginx location that should be used for getting the DRM info for the file.

vod_drm_info_cache

Configures the size and shared memory object name of the drm info cache.

vod_drm_request_uri

Sets the uri of drm info requests, the parameter value can contain variables. In case of multi url, $vod_suburi will be the current sub uri (a separate drm info request is issued per sub URL)

vod_min_single_nalu_per_frame_segment

Sets the minimum segment index (1-based) that should be assumed to have a single h264 nalu per frame. If the value is 0, no assumption is being made on the number of nal units per frame. This setting only affects DASH and MSS configurations that have DRM enabled.

When transcoding videos using libx264, by default, all frames have a single nal unit, except the first frame that contains an additional nalu with the libx264 copyright information. Setting this parameter to a value greater than 0 can provide a significant performance improvement, since the layout of the segment can be calculated in advance, allowing the module to:

Configuration directives - DASH

vod_dash_absolute_manifest_urls

When enabled the server returns absolute URLs in MPD requests

vod_dash_manifest_file_name_prefix

The name of the MPD file (an mpd extension is implied).

vod_dash_profiles

Sets the profiles that are returned in the MPD tag in manifest responses.

vod_dash_init_file_name_prefix

The name of the MP4 initialization file (an mp4 extension is implied).

vod_dash_fragment_file_name_prefix

The name of the fragment files (an m4s extension is implied).

vod_dash_manifest_format

Sets the MPD format, available options are:

vod_dash_subtitle_format

Sets the format of the subtitles returned in the MPD, available options are:

vod_dash_init_mp4_pssh

When enabled, the DRM pssh boxes are returned in the DASH init segment and in the manifest. When disabled, the pssh boxes are returned only in the manifest.

vod_dash_duplicate_bitrate_threshold

The bitrate threshold for removing identical bitrates, streams whose bitrate differences are less than this value will be considered identical.

vod_dash_use_base_url_tag

When enabled, a BaseURL tag will be used to specify the fragments/init segment base url. Otherwise, the media/initialization attributes under SegmentTemplate will contain absolute URLs.

Configuration directives - HDS

vod_hds_absolute_manifest_urls

When enabled the server returns the base URL in the F4M manifest

vod_hds_manifest_file_name_prefix

The name of the HDS manifest file (an f4m extension is implied).

vod_hds_fragment_file_name_prefix

The prefix of fragment file names, the actual file name is frag-f<file-index>-v<video-track-index>-a<audio-track-index>-Seg1-Frag<index>.

vod_hds_generate_moof_atom

When enabled the module generates a moof atom in the HDS fragments, when disabled only an mdat atom is generated. Turning this parameter off reduces the packaging overhead, however the default is on since Adobe tools are generating this atom.

Configuration directives - HLS

vod_hls_encryption_method

Sets the encryption method of HLS segments, allowed values are: none (default), aes-128, sample-aes, sample-aes-cenc.

vod_hls_force_unmuxed_segments

When enabled the server returns the audio stream in separate segments than the ones used by the video stream (using EXT-X-MEDIA)

vod_hls_container_format

Sets the container format of the HLS segments. The default behavior is to use fmp4 for HEVC, and mpegts otherwise (Apple does not support HEVC over MPEG TS).

vod_hls_absolute_master_urls

When enabled the server returns absolute playlist URLs in master playlist requests

vod_hls_absolute_index_urls

When enabled the server returns absolute segment URLs in media playlist requests

vod_hls_absolute_iframe_urls

When enabled the server returns absolute segment URLs in iframe playlist requests

vod_hls_output_iframes_playlist

When disabled iframe playlists are not returned as part of master playlists

vod_hls_master_file_name_prefix

The name of the HLS master playlist file (an m3u8 extension is implied).

vod_hls_index_file_name_prefix

The name of the HLS media playlist file (an m3u8 extension is implied).

vod_hls_iframes_file_name_prefix

The name of the HLS I-frames playlist file (an m3u8 extension is implied).

vod_hls_segment_file_name_prefix

The prefix of segment file names, the actual file name is seg-<index>-v<video-track-index>-a<audio-track-index>.ts.

vod_hls_init_file_name_prefix

The name of the init segment file name, only relevant when using fmp4 container.

vod_hls_encryption_key_file_name

The name of the encryption key file name, only relevant when encryption method is not none.

vod_hls_encryption_key_uri

Sets the value of the URI attribute of EXT-X-KEY, only relevant when encryption method is not none. The parameter value can contain variables.

vod_hls_encryption_key_format

Sets the value of the KEYFORMAT attribute of EXT-X-KEY, only relevant when encryption method is not none.

vod_hls_encryption_key_format_versions

Sets the value of the KEYFORMATVERSIONS attribute of EXT-X-KEY, only relevant when encryption method is not none.

vod_hls_mpegts_interleave_frames

When enabled, the HLS muxer interleaves frames of different streams (audio / video). When disabled, on every switch between audio / video the muxer flushes the MPEG TS packet.

vod_hls_mpegts_align_frames

When enabled, every video / audio frame is aligned to MPEG TS packet boundary, padding is added as needed.

vod_hls_mpegts_output_id3_timestamps

When enabled, an ID3 TEXT frame is outputted in each TS segment. The content of the ID3 TEXT frame can be set using the directive vod_hls_mpegts_id3_data.

vod_hls_mpegts_id3_data

Sets the data of the ID3 TEXT frame outputted in each TS segment, when vod_hls_mpegts_output_id3_timestamps is set to on. When the directive is not set, the ID3 frames contain by default a JSON object of the format {"timestamp":1459779115000,"sequenceId":"{id}"}:

vod_hls_mpegts_align_pts

When enabled, the module will shift back the dts timestamps by the pts delay of the initial frame. This can help keep the pts timestamps aligned across multiple renditions.

vod_hls_encryption_output_iv

When enabled, the module outputs the IV attribute in returned #EXT-X-KEY tags.

Configuration directives - MSS

vod_mss_manifest_file_name_prefix

The name of the manifest file (has no extension).

vod_mss_duplicate_bitrate_threshold

The bitrate threshold for removing identical bitrates, streams whose bitrate differences are less than this value will be considered identical.

Configuration directives - thumbnail capture

vod_thumb_file_name_prefix

The name of the thumbnail file (a jpg extension is implied).

vod_thumb_accurate_positioning

When enabled, the module grabs the frame that is closest to the requested offset. When disabled, the module uses the keyframe that is closest to the requested offset. Setting this parameter to off can result in faster thumbnail capture, since the module always decodes a single video frame per request.

vod_gop_look_behind

Sets the interval (in milliseconds) before the thumbnail offset that should be loaded. This setting should be set to the maximum GOP size, setting it to a lower value may result in capture failure. Note that the metadata of all frames between offset - vod_gop_look_behind and offset + vod_gop_look_ahead is loaded, however only the frames of the minimum GOP containing offset will be read and decoded.

vod_gop_look_ahead

Sets the interval (in milliseconds) after the thumbnail offset that should be loaded.

Configuration directives - volume map

vod_volume_map_file_name_prefix

The name of the volume map file (a csv extension is implied).

vod_volume_map_interval

Sets the interval/resolution (in milliseconds) of the volume map.

Configuration directives - misc

vod_ignore_edit_list

When enabled, the module ignores any edit lists (elst) in the MP4 file.

vod_parse_hdlr_name

When enabled, the module parses the name field of the hdlr MP4 atom, and uses it as the stream label.

vod_parse_udta_name

When enabled, the module parses the name atom child of the udta MP4 atom, and uses it as the stream label.

Nginx variables

The module adds the following nginx variables:

Note: Configuration directives that can accept variables are explicitly marked as such.

Sample configurations

Local configuration

    http {
        upstream fallback {
            server fallback.kaltura.com:80;
        }

        server {
            # vod settings
            vod_mode local;
            vod_fallback_upstream_location /fallback;
            vod_last_modified 'Sun, 19 Nov 2000 08:52:00 GMT';
            vod_last_modified_types *;

            # vod caches
            vod_metadata_cache metadata_cache 512m;
            vod_response_cache response_cache 128m;

            # gzip manifests
            gzip on;
            gzip_types application/vnd.apple.mpegurl;

            # file handle caching / aio
            open_file_cache          max=1000 inactive=5m;
            open_file_cache_valid    2m;
            open_file_cache_min_uses 1;
            open_file_cache_errors   on;
            aio on;

            location ^~ /fallback/ {
                internal;
                proxy_pass http://fallback/;
                proxy_set_header Host $http_host;
            }

            location /content/ {
                root /web/;
                vod hls;

                add_header Access-Control-Allow-Headers '*';
                add_header Access-Control-Expose-Headers 'Server,range,Content-Length,Content-Range';
                add_header Access-Control-Allow-Methods 'GET, HEAD, OPTIONS';
                add_header Access-Control-Allow-Origin '*';
                expires 100d;
            }
        }
    }

Mapped configuration

    http {
        upstream kalapi {
            server www.kaltura.com:80;
        }

        upstream fallback {
            server fallback.kaltura.com:80;
        }

        server {
            # vod settings
            vod_mode mapped;
            vod_upstream_location /kalapi;
            vod_upstream_extra_args "pathOnly=1";
            vod_fallback_upstream_location /fallback;
            vod_last_modified 'Sun, 19 Nov 2000 08:52:00 GMT';
            vod_last_modified_types *;

            # vod caches
            vod_metadata_cache metadata_cache 512m;
            vod_response_cache response_cache 128m;
            vod_mapping_cache mapping_cache 5m;

            # gzip manifests
            gzip on;
            gzip_types application/vnd.apple.mpegurl;

            # file handle caching / aio
            open_file_cache          max=1000 inactive=5m;
            open_file_cache_valid    2m;
            open_file_cache_min_uses 1;
            open_file_cache_errors   on;
            aio on;

            location ^~ /fallback/ {
                internal;
                proxy_pass http://fallback/;
                proxy_set_header Host $http_host;
            }

            location ^~ /kalapi/ {
                internal;
                proxy_pass http://kalapi/;
                proxy_set_header Host $http_host;
            }

            location ~ ^/p/\d+/(sp/\d+/)?serveFlavor/ {
                # encrypted hls
                vod hls;
                vod_secret_key "mukkaukk$vod_filepath";
                vod_hls_encryption_method aes-128;

                add_header Access-Control-Allow-Headers '*';
                add_header Access-Control-Expose-Headers 'Server,range,Content-Length,Content-Range';
                add_header Access-Control-Allow-Methods 'GET, HEAD, OPTIONS';
                add_header Access-Control-Allow-Origin '*';
                expires 100d;
            }
        }
    }

Mapped + Remote configuration

    http {
        upstream jsonupstream {
            server jsonserver:80;
        }

        server {
            # vod settings
            vod_mode mapped;
            vod_upstream_location /json;
            vod_remote_upstream_location /proxy;
            vod_upstream_extra_args "pathOnly=1";
            vod_last_modified 'Sun, 19 Nov 2000 08:52:00 GMT';
            vod_last_modified_types *;

            # vod caches
            vod_metadata_cache metadata_cache 512m;
            vod_response_cache response_cache 128m;
            vod_mapping_cache mapping_cache 5m;

            # gzip manifests
            gzip on;
            gzip_types application/vnd.apple.mpegurl;

            # file handle caching / aio
            open_file_cache   max=1000 inactive=5m;
            open_file_cache_valid    2m;
            open_file_cache_min_uses 1;
            open_file_cache_errors   on;
            aio on;

            location ^~ /json/hls/ {
                internal;
                proxy_pass http://jsonupstream/;
                proxy_set_header Host $http_host;
            }

            location ~ /proxy/([^/]+)/(.*) {
                internal;
                proxy_pass $1://$2;
                resolver 8.8.8.8;
            }

            location ~ ^/hls/ {
                vod hls;

                add_header Access-Control-Allow-Headers '*';
                add_header Access-Control-Expose-Headers 'Server,range,Content-Length,Content-Range';
                add_header Access-Control-Allow-Methods 'GET, HEAD, OPTIONS';
                add_header Access-Control-Allow-Origin '*';
                expires 100d;
            }
        }
    }

Set it up so that http://jsonserver:80/test.json returns the following JSON:

    {
        "sequences": [{
            "clips": [{
                "type": "source",
                "path": "/http/commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4"
            }]
        }]
    }

And use this stream URL - http://nginx-vod-server/hls/test.json/master.m3u8

Remote configuration

    http {
        upstream kalapi {
            server www.kaltura.com:80;
        }

        server {
            # vod settings
            vod_mode remote;
            vod_upstream_location /kalapi;
            vod_last_modified 'Sun, 19 Nov 2000 08:52:00 GMT';
            vod_last_modified_types *;

            # vod caches
            vod_metadata_cache metadata_cache 512m;
            vod_response_cache response_cache 128m;

            # gzip manifests
            gzip on;
            gzip_types application/vnd.apple.mpegurl;

            location ^~ /kalapi/ {
                internal;
                proxy_pass http://kalapi/;
                proxy_set_header Host $http_host;
            }

            location ~ ^/p/\d+/(sp/\d+/)?serveFlavor/ {
                vod hls;

                add_header Access-Control-Allow-Headers '*';
                add_header Access-Control-Expose-Headers 'Server,range,Content-Length,Content-Range';
                add_header Access-Control-Allow-Methods 'GET, HEAD, OPTIONS';
                add_header Access-Control-Allow-Origin '*';
                expires 100d;
            }
        }
    }

Copyright & License

All code in this project is released under the AGPLv3 license unless a different license for a particular library is specified in the applicable library path.

Copyright © Kaltura Inc. All rights reserved.