search

bondagit / aes67-linux-daemon

AES67 Linux Daemon with configuration WebUI
GNU General Public License v3.0
372 stars 82 forks source link
aes67 audio audio-over-ip rtp rtp-streaming

readme

AES67 Linux Daemon

AES67 Linux Daemon is a Linux implementation of AES67 interoperability standard used to distribute and synchronize real time audio over Ethernet. See https://en.wikipedia.org/wiki/AES67 for additional info.

Status

Daemon tests status

WebUI release status

Introduction

The daemon is a Linux process that uses the Merging Technologies ALSA RAVENNA/AES67 Driver to handle PTP synchronization and RTP streams and exposes a REST interface for configuration and status monitoring.

The ALSA AES67 Driver implements a virtual ALSA audio device that can be configured using Sources and Sinks and it's clocked using the PTP clock. A Source reads audio samples from the ALSA playback device and sends RTP packets to a configured multicast or unicast address. A Sink receives RTP packets from a specific multicast or unicast address and writes them to the ALSA capture device.

A user can use the ALSA capture device to receive synchronized incoming audio samples from an RTP stream and the ALSA playback device to send synchronized audio samples to an RTP stream. The binding between a Source and the ALSA playback device is determined by the channels used during the playback and the configured Source channels map. The binding between a Sink and the ALSA capture device is determined by the channels used while recoding and the configured Sink channels map.

The driver handles the PTP and RTP packets processing and acts as a PTP clock slave to synchronize with a master clock on the specified PTP domain. All the configured Sources and Sinks are synchronized using the same PTP clock.

The daemon communicates with the driver for control, configuration and status monitoring only by using netlink sockets. The daemon implements a REST interface to configure and monitor the Sources, the Sinks and PTP slave. See README for additional info. It also implements SAP sources discovery and advertisement compatible with AES67 standard and mDNS sources discovery and advertisement compatible with Ravenna standard.

A WebUI is provided to allow daemon and driver configuration and monitoring. The WebUI uses the daemon REST API and exposes all the supported configuration paramaters for the daemon, the PTP slave clock, the Sources and the Sinks. The WebUI can also be used to monitor the PTP slave status and the Sinks status and to browse the remote SAP and mDNS sources.

License

AES67 daemon and the WebUI are licensed under GNU GPL.

The daemon uses the following open source:

Prerequisite

The daemon and the test have been verified starting from Ubuntu 18.04 distro onwards for ARMv7 and x86 platforms using:

The following platforms have been used for testing:

A NanoPi NEO2 with Allwinner H5 Quad-core 64-bit Cortex A53 processor. See Armbian NanoPi NEO2 for additional information about how to setup Ubuntu on this board.

A Mini PC N40 with Intel® Celeron® Processor N4020 , 2 Cores/2 Threads (4M Cache, up to 2.80 GHz). See Minisforum N40 Mini PC and how to Install Ubuntu on a fanless Mini PC.

The ubuntu-packages.sh script can be used to install all the packages required to compile and run the AES67 daemon, and the platform compatibility test.

Important CPU scaling events could affect daemon streams causing unexpected distortions, see CPU scaling events and scripts notes.

Important Starting from Linux kernel 5.10.x onwards a change in a kernel parameter is required to fix a problem with round robin scheduler causing the latency test to fail, see Real Time Scheduler Throttling.

Important PulseAudio must be disabled or uninstalled for the daemon to work properly, see PulseAudio and scripts notes.

How to build and setup the daemon

Install on your target machine (both ARM or X86) an Ubuntu Linux distribution. NO Virtual machines. The daemon should work on all Ubuntu starting from 18.04 onward, it's possible to use other distros.

Devices and interoperability tests

See Devices and interoperability tests with the AES67 daemon

HTTP Streamer

The HTTP Streamer was introduced with the daemon version 2.0 and it is used to receive AES67 audio streams via HTTP file streaming.

The HTTP Streamer can be enabled via the _streamerenabled daemon parameter. When the Streamer is active the daemon starts capturing the configured Sinks up to the maximum number of channels configured by the _streamerchannels parameters. The captured PCM samples are split into _streamer_filesnum files of _streamer_fileduration duration (in seconds) for each sink, compressed using AAC LC codec and served via HTTP. Screenshot 2024-06-15 at 15 36 48 The HTTP streamer requires the libfaac-dev package to compile.

Please note that since the HTTP Streamer uses the RAVENNA ALSA device for capturing it's not possible to use such device for other audio captures.

AES67 USB Receiver and Transmitter

See Use your board as AES67 USB Receiver and Transmitter

Repository content

daemon directory

This directory contains the AES67 daemon source code. The daemon can be cross-compiled for multiple platforms and implements the following functionalities:

See the README file in this directory for additional information about the AES67 daemon configuration and the daemon HTTP REST API.

The directory also contains the daemon tests in the tests subdirectory.

Daemon tests can be executed via a Docker container by using a fake version of the daemon driver manager. This was implemented to perfom automated execution of the tests via a GitHub workflow.

To build the Docker image for the daemon regression tests run:

  docker build --progress=plain -f ./Dockerfile.daemon_tests -t aes67-daemon-tests  .

To run the tests:

  docker run aes67-daemon-tests

webui directory

This directory contains the AES67 daemon WebUI configuration implemented using React. With the WebUI a user can do the following operations:

3rdparty directory

This directory is used to download the 3rdparty open source. The aes67-daemon branch of ravenna-alsa-lkm repository contains the ALSA RAVENNA/AES67 module code plus all the patches applied to it.

The ALSA RAVENNA/AES67 kernel module is responsible for:

See ALSA RAVENNA/AES67 Driver README for additional information about the Merging Technologies module and for proper Linux Kernel configuration and tuning.

systemd directory

This directory contains systemd configuration files for the daemon.

The daemon integrates with systemd watchdog.

To enable it recompile the daemon with the CMake option _-DWITHSYSTEMD=ON

You can install the daemon under systemd by using the script systemd/install.sh:

cd systemd
sudo ./install.sh

Before starting the daemon edit /etc/daemon.conf and make sure the _interfacename parameter is set to your ethernet interface.

To start the daemon use:

 sudo systemctl start aes67-daemon

To stop it use:

 sudo systemctl stop aes67-daemon

The daemon requires the MergingRavennaALSA module to run.

You can usally install the module using the following commands:

cd 3rdparty/ravenna-alsa-lkm/driver
sudo make modules_install

If this doesn't work because you miss kernel certificate follow the instructions at: No OpenSSL sign-file signing_key.pem

Finally use the command to load the modules:

sudo depmod -a

test directory

This directory contains the files used to run the daemon platform compatibility test on the network loopback interface. The test is described below.

Run the platform compatibility test

Before attempting to use the AES67 daemon on a specific host or board it's highly recommended to run the platform test. The platform test runs a playback and recording of an RTP session on the loopback network device using the ALSA RAVENNA/AES67 modules and checks that no audio samples get corrupted or lost. This test can be executed using the run_test.sh script. See script notes.

The script allows a user to test a specific configuration and it can be used to ensure that the daemon will be able to operate smoothly with such config on the current platform.

  Usage run_test.sh sample_format sample_rate channels duration
       sample_format can be one of S16_LE, S24_3LE, S32_LE
       sample_rate can be one of 44100, 48000, 88200, 96000, 192000, 384000
       channels can be one of 2, 4, 6, up to 64
       duration is in the range 1 to 10 minutes

For example to test the typical AES67 configuration run:

  ./run_test.sh S24_3LE 48000 2 5

The test performs the following operations:

If the test result is OK it means that the selected configuration can run smoothly on your platform.

A 64 channels configuration was succesfully tested on the Mini PC with Intel Celeron N4020 processor.

A 24 channels configuration was succesfully tested on the NanoPi NEO2 board.

If the test reports a failure you may try to stop all the possible additional loads running on the host and repeat it. If after this the test fails systematically it means you cannot achieve a good reliability with the specified configuration. In this case you may try to configure a different driver timer basic tick period in the daemon configuration file (parameter tic_frame_size_at_1fs in test/daemon.conf). By default this parameter is set to 48 (1ms latency) and the valid range is from 48 to 480 with steps of 48. Note that higher values of this parameter (values above 48) lead to higher packets processing latency and this breaks the compatibility with certain devices.

Run the platform latency test

The platform latency test can be used to measure the minimum end-to-end latency that can be achieved on a specific platform.

The test is based on the same setup used for the platform compatibility test where a Sink is configured to receive audio samples from a Source both running on the same device using the network loopback interface.

A test application plays audio samples on the RAVENNA playback device and measures the time till the samples are received on the RAVENNA capture device.

The test setup and the end-to-end latency measured are shown in the picture below:

image

The test can be executed using the run_latency_test.sh script. See script notes.

The script allows a user to test the latency on a specific configuration and it can be used to ensure that a specific ALSA buffer size can be used:

  Usage run_latency_test.sh sample_format sample_rate channels duration frames
       sample_format can be one of S16_LE, S24_3LE, S32_LE
       sample_rate can be one of 44100, 48000, 88200, 96000, 192000, 384000
       channels can be one of 2, 4, 6, up to 64
       duration of the test in seconds
       frames buffer size in frames

The specified buffer size in frames starts from _tic_frame_size_at1fs * 2 (96 by default) in steps of _tic_frame_size_at1fs.

For example, to test the typical AES67 configuration for 1 minute and a buffer size of 96 frames run:

  ./run_latency_test.sh S24_3LE 48000 2 60 96

If no underrun errors occurred during the test the requested buffer size can be used and the end-to-end latency measured is printed at the end:

  Trying latency 96 frames, 2000.000us, 2.000000ms (500.0000Hz)
  Success
  Playback:
  *** frames = 480096 ***
    state       : RUNNING
    trigger_time: 170.196886
    tstamp      : 0.000000
    delay       : 96
    avail       : 0
    avail_max   : 48
  Capture:
  *** frames = 480000 ***
    state       : RUNNING
    trigger_time: 170.196887
    tstamp      : 0.000000
    delay       : 0
    avail       : 0
    avail_max   : 48
  Maximum read: 48 frames
  Maximum read latency: 1000.000us, 1.000000ms (1000.0000Hz)
  Playback time = 170.196886, Record time = 170.196887, diff = -1
  End to end latency: 5.999 msecs
  Terminating processes ...
  daemon exiting with code: 0

The previous test was run on a NanoPi NEO2 board with Ubuntu distro.

A 64 channels configuration was succesfully tested on the Mini PC with Intel Celeron N4020 processor.

A 24 channels configuration was succesfully tested on the NanoPi NEO2 board.

In case underrun happened the status reported is:

     state       : XRUN

and the specified buffer size cannot be used.

Important Starting from Linux kernel 5.10.x onwards a change in a kernel parameter is required to fix a problem with round robin scheduler causing the latency test to fail, see Real Time Scheduler Throttling.

Run the daemon tests

To run daemon tests install the ALSA RAVENNA/AES67 kernel module with:

  sudo insmod 3rdparty/ravenna-alsa-lkm/driver/MergingRavennaALSA.ko

setup the kernel parameters with:

  sudo sysctl -w net/ipv4/igmp_max_memberships=66

make sure that no instances of the aes67-daemon are running, enter the tests subdirectory and run:

  ./daemon-test -p

Notes