search

emarsden / dash-mpd-cli

Download media content from a DASH-MPEG or DASH-WebM MPD manifest
https://emarsden.github.io/dash-mpd-cli/
MIT License
331 stars 34 forks source link
dash downloader mpeg mpeg-dash rust streaming video

readme

dash-mpd-cli

A commandline application for downloading media content from a DASH MPD file, as used for on-demand replay of TV content and video streaming services.

Crates.io CI Container size Dependency status LICENSE [Github downloads]()

Terminal capture

DASH (dynamic adaptive streaming over HTTP), also called MPEG-DASH, is a technology used for media streaming over the web, commonly used for video on demand (VOD) and “replay/catch-up TV” services. The Media Presentation Description (MPD) is an XML document that lists the resources (manifest or “playlist”) forming a streaming service. A DASH client uses the manifest to determine which assets to request in order to perform adaptive streaming of the content. DASH MPD manifests can be used with content using different codecs (including H264, HEVC, AV1, AAC, VP9, MP4A, MP3) and containers (MP4, WebM, Matroska, AVI). There is a good explanation of adaptive bitrate video streaming at howvideo.works.

This commandline application allows you to download streaming video or audio to your local device. This involves selecting the alternative with the most appropriate encoding (in terms of bitrate, codec, etc.), fetching segments of the content using HTTP or HTTPS requests and muxing audio and video segments together. There is also support for downloading subtitles (mostly WebVTT, TTML, SRT, tx3g and SMIL formats, with some support for wvtt format).

This application builds on the dash-mpd crate. It works for streaming that uses DASH adaptive streaming (MPD manifests), and doesn’t currently have support for HLS streaming (m3u8 manifests).

📖 You may be interested in the user manual.

Features

The following features are supported:

The following are not currently supported:

Run safely in a Docker container

The application, alongside the external helper applications that it uses for muxing media streams, for extracting/converting subtitle streams, and for decrypting content infected with DRM, are available as a prebuilt container, which is probably the easiest and safest way to run it. The container is packaged with a minimal Alpine Linux installation and can be run on any host that can run Linux containers (using Podman or Docker on Linux, Microsoft Windows and MacOS). It’s available in the GitHub Container Registry ghcr.io.

[!TIP] What are the advantages of running in a container, instead of natively on your machine?

  • Much safer, because the container isn't able to modify your host machine, except for writing downloaded media to the directory you specify. This is a very good idea when running random software you downloaded from the internet!

  • No need to install the various helper applications (ffmpeg, mkvmerge, mp4decrypt, shaka-packager, MP4Box, xsltproc), which are already present in the container.

  • Automatically run the latest version of dash-mpd-cli and the various helper applications (the container runtime will pull the latest version for you automatically).

  • Podman and Docker also allow you to set various limits on the resources allocated to the container (number of CPUs, memory); see their respective documentation.

Unlike running software in a virtual machine, there is only a negligeable performance penalty to running in a container. That’s not quite true: if you’re running the container on an aarch64 (“Apple Silicon”) Mac, Podman will set up a virtual machine for you. On Windows, Podman will set up a low-overhead WSL2 virtual machine for you.

I recommend installing Podman because it’s fully free software, whereas Docker is partly commercial. Podman is also able to run containers “rootless”, without special privileges, which is good for security.

To run the container with podman:

podman machine start (optional step, only required on Windows and MacOS)
podman run --rm -v .:/content ghcr.io/emarsden/dash-mpd-cli -v <MPD-URL> -o foo.mp4

On the first run, this will fetch the container image (around 220 MB) from the GitHub Container Registry ghcr.io, and will save it on your local disk for later uses. You can later delete the image if you no longer need it using podman image rm and the image id shown by podman images (see the user manual for details).

📁 Your current working directory (.) will be mounted in the container as /content, which will be the working directory in the container. This means that an output file specified without a root directory, such as foo.mp4, will be saved to your current working directory on the host machine.

On Linux/AMD64, it’s also possible to run the container using the gVisor container runtime runsc, which uses a sandbox to further improve security (strong isolation, protection against privilege escalation). This requires installation of runsc and running as root (runsc doesn’t currently support rootless operation).

sudo apt install runsc
sudo podman --runtime=runsc run --rm -v .:/content ghcr.io/emarsden/dash-mpd-cli -v <MPD-URL> -o foo.mp4

The container image is a multiarch manifest, currently built for the following platforms:

Installation

If you prefer to install the software and its dependencies on your computer in the traditional way, you can download a prebuilt binary or build from source yourself.

Binary releases are available on GitHub for GNU/Linux on AMD64 (statically linked against Musl Libc to avoid glibc versioning problems), Microsoft Windows on AMD64 and MacOS on aarch64 (“Apple Silicon”). These are built automatically on the GitHub continuous integration infrastructure.

You can also build from source using an installed Rust development environment:

cargo install dash-mpd-cli

This installs the binary to your installation root’s bin directory, which is typically $HOME/.cargo/bin.

You should also install the following dependencies:

This crate is tested on the following platforms:

Usage

See the 📖 user manual.

Muxing

The underlying library dash-mpd-rs has two methods for muxing audio and video streams together. If the library feature libav is enabled (which is not the default configuration), muxing support is provided by ffmpeg’s libav library, via the ac_ffmpeg crate. Otherwise, muxing is implemented by calling an external muxer, mkvmerge (from the MkvToolnix suite), ffmpeg, vlc or MP4Box as a subprocess. Note that these commandline applications implement a number of checks and workarounds to fix invalid input streams that tend to exist in the wild. Some of these workarounds are implemented here when using libav as a library, but not all of them, so download support tends to be more robust with the default configuration (using an external application as a subprocess). The libav feature currently only works on Linux.

The choice of external muxer depends on the filename extension of the path supplied to --output or -o (which will be .mp4 if you don't specify the output path explicitly):

You can specify a different order of preference for muxing applications using the --muxer-preference commandline option. For example, --muxer-preference avi:vlc,ffmpeg means that for an AVI media container the external muxer vlc will be tried first, then ffmpeg in case of failure. This commandline option can be used multiple times to specify options for different container types.

Similar tools

Similar commandline tools that are able to download content from a DASH manifest:

However, dash-mpd-cli (this application) is able to download content from certain streams that do not work with other applications:

Building

$ sudo apt install protobuf-compiler
$ git clone https://github.com/emarsden/dash-mpd-cli
$ cd dash-mpd-cli
$ cargo build --release
$ target/release/dash-mpd-cli --help

The application can also be built statically with the musl-libc target on Linux. First install the MUSL C standard library on your system. Add linux-musl as a target to your Rust toolchain, then rebuild for the relevant target:

$ sudo apt install musl-dev
$ rustup target add x86_64-unknown-linux-musl
$ cargo build --release --target x86_64-unknown-linux-musl

Static musl-libc builds don’t work with OpenSSL, which is why we disable default features on the dash-mpd crate and build it with rustls support (a Rust TLS stack). You may encounter some situations where rustls fails to connect (handshake errors, for example) but other applications on your system can connect. These differences in behaviour are typically due to different configurations for the set of root certificates. If you prefer to use your machine’s native TLS stack, replace both instances of rustls-tls by native-tls in Cargo.toml and rebuild.

Why?

The dash-mpd-rs library at the core of this application was developed to allow the author to watch a news programme produced by a public media broadcaster whilst at the gym. The programme is published as a DASH stream on the broadcaster’s “replay” service, but network service at the gym is sometimes poor. First world problems!

[!WARNING] The author is not the morality police nor a lawyer, but please note that redistributing media content that you have not produced may, depending on the publication licence, be a breach of intellectual property laws. Also, circumventing DRM may be prohibited in some countries.

License

This project is licensed under the MIT license. For more information, see the LICENSE-MIT file.