EnvelopSound / Earshot

Containerized RTMP->DASH transcoder for live-streaming Higher-Order Ambisonic and other multichannel content
GNU General Public License v2.0
67 stars 7 forks source link

Earshot -- Envelop Ambisonic RTMP Streaming Higher-Order Transcoder

CI

Earshot Screenshot

Envelop Ambisonic RTMP Streaming Higher-Order Transcoder (Earshot) is a containerized multichannel RTMP->DASH transcoder, based on ngnix. Earshot can be used to transcode higher-order Ambisonics and other multichannel live streams for the web.

About Envelop

Envelop is a nonprofit organization that amplifies the connective power of music through immersive listening spaces and open source spatial audio software. Three-dimensional experiences of sound and music bring people together, catalyzing shared moments of inspiration, empathy and wonder.

Motivation for Earshot

Tools such as Envelop for Live have made it easy for musicians to compose and perform ambisonic content. However, there remain limited options for live streaming ambisonic content, particularly beyond first order. Ambisonic live streaming has applications for VR/AR/XR and immersive home listening experiences.

Earshot can be used in combination with pkviet's OBS Studio Music Edition which supports multichannel AAC encoding up to 16.0. For more information on OBS see obsproject.com.

Earshot is designed to be easily deployed to a cloud-based hosting solution, such as AWS ECS, DigitalOcean, etc.

Earshot is GPL licensed. However, if you are interested in a commercial license for Earshot, please contact us at envelop@envelop.us.

Features

Key Technologies Used

Known Limitations

Running Earshot Locally

Requirements:

NOTE: Earshot contains binaries that are inflated with Git LFS. You must install Git LFS first, then git clone this repository. Downloading Earshot as a zip file of any kind (master or a release) is not supported and will not work.

Optional Requirements:

To generate a multichannel audio stream in a format such as third-order AMBIX you can use any of the following:

Setup with OBS:

1. Create your environment settings

To setup your environment, first create a copy of the .env.example file. For example:

cp .env.example .env

Please update the environmental variables in the .env file as needed.

Note: if you are using SSL please be sure to add the domain and email fields you will be serving Earshot from. Also, if you will be running tests, please note that the tests will use "stream1" as the stream key. If you are using a custom authentication server (please see RTMP Authentication section for more on this) and will be using tests, please fill out RTMP_AUTH_TOKEN so that it holds the auth token for the stream key: "stream1".

2. Build and run the Docker container for the transcoder

From this (project root) directory:

docker-compose up --build nginx-rtmp

3. Open OBS Music Edition

Click Settings and set the following:

Stream

Output

Audio

4. Select Audio Source

5. Start Streaming

Click Start Streaming in OBS

Your DASH stream manifest is now available at http://localhost/dash/stream1.mpd

DASH stream webtools are available under http://localhost/webtools

RTMP Authentication

To add a RTMP auth secret token you can update the "RTMP_AUTH_TOKEN" environment variable in the docker-compose.yml or .env file, e.g. - RTMP_AUTH_TOKEN=my_secret.

In order to prevent brute force attacks, Earshot will limit authentication requests to one per second. To allow for flexibility and legitimate traffic surges, occasional bursts for up to two requests are accepted, until, on avarage, the one per second limit is retained.

To use a custom http auth server, you can update the RTMP_AUTH_URL environment variable. Earshot will pass name (the Stream Key), token (the auth token provided by the user), and other parameters to your server. You can read the full list of parameters.

On your streaming client, appent the secret using the token GET parameter to the request.

image

FFMPEG Flags

If you want to add additional flags for ffmpeg that is called within the transcoder -- for example, more adaptive DASH stream bitrates -- you can update the "FFMPEG_FLAGS" environment variable in the docker-compose.yml.

Troubleshooting

You may be missing key binaries installed by Git LFS. Make sure you have used git clone to get Earshot, not downloaded it a zip file. If you installed Git LFS after cloning this repository, you can run git lfs checkout to inflate the binaries.

You can look at the nginx-rtmp ffmpeg logs in the container. First open a shell:

docker exec -it earshot_nginx-rtmp_1 sh

Then tail the log file here:

tail -f /tmp/nginx_rtmp_ffmpeg_log

Using Webtools

Load http://localhost/webtools in your browser to monitor and debugs streams from the transcoder. Chrome or Firefox preferred.

To test different Dash.js Player settings, edit the JSON in the Dash.js settings box. Any changes applied will affect the live stream on your browser in real time, or reload the new URL -- containing the encoded settings as a parameter -- to test the player settings loaded initially on page load.

Known Webtools issues:

Deploying Earshot to AWS CloudFormation

Earshot can be easily deployed to AWS using AWS CLI and CloudFormation. You can customize the deployment configuration in the CloudFormation template file in the templates/ directory

A sample configuration for adaptive bandwidth streaming can be found in the cloudformation-template-adaptive.yaml file with a c5n.2xlarge EC2 instance type. With an 1080p input stream with additional 720p and 360p adaptive resolutions and 16 channels at 128k each, CPU usage was found to be at 33% with no clients for this instance type.

Deploying the stack

Before deploying, you will first need to generate an EC2 Key Pair.

aws cloudformation create-stack --region=us-west-2 --stack-name earshot-stack --template-body file://templates/cloudformation-template.yaml --parameters ParameterKey=InstanceType,ParameterValue=t3.micro ParameterKey=KeyName,ParameterValue=<YOUR_KEY_PAIR_NAME> --capabilities CAPABILITY_IAM

Accessing the public URL

To get the public URL of your Earshot instance:

  1. Go to AWS Console -> ECS
  2. Open cluster named earshot-stack-EcsCluster
    • note: full cluster name may appear be longer. For example: earshot-stack-EcsCluster-o3KvtIz8VZTt
  3. Open the "ECS Instances" tab
  4. Click into the first ECS instance
  5. Copy the "Public DNS" field

RTMP Access

This is the stream URL you should use for OBS or whichever live streaming application you are using at the source, e.g.: rtmp://<YOUR_INSTANCE_PUBLIC_DNS>:1935/live/stream1

Webtools Access

To access Webtools, navigate to: http://<YOUR_INSTANCE_PUBLIC_DNS>/webtools

Custom FFmpeg Flags and RTMP Auth

The ECS container currently uses predefined values for RTMP authentication and FFmpeg flags.

If you want to change the container environment variables you can change the stack variables when you deploy it. Or if you want to make changes live, you can update the task definition environment in the AWS console.

CloudFormation parameters

RtmpAuthToken
This is your RTMP authentication token
FfmpegFlags
Add additional flags to FFMPEG

Deploying with custom parameters

aws cloudformation create-stack --region=us-west-2 --stack-name earshot-stack --template-body file://templates/cloudformation-template.yaml --parameters ParameterKey=InstanceType,ParameterValue=t3.micro ParameterKey=KeyName,ParameterValue=<YOUR_KEY_PAIR_NAME> ParameterKey=RtmpAuthToken,ParameterValue=<YOUR_CUSTOM_AUTH_TOKEN> ParameterKey=FfmpegFlags,ParameterValue="-loglevel repeat+level+verbose" --capabilities CAPABILITY_IAM

HTTPS / SSL

Google Chrome, amongst other browers, requires HTTPS to serve MPEG DASH streams.

One way to achieve this in a production environment is to wrap Earshot in a CDN like Amazon Cloudfront. Cloudfront will serve requests over HTTP and can interface with the Earshot server via plain old HTTP (the "origin"). You can read more on how to achieve this with CloudFront [here].(https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/live-streaming.html)

If you want to run Earshot to serve HTTPS without a CDN, check out the feature/ssl branch.

Setup Route53

Adding a domain to your earshot service is an optional step. However, there are many benefits to it, such as HTTPS support and more.

Route53 requirements

To use the Route53 Cloudformation stack, you must have already purchased a domain from AWS.

For more info on registering Route53 domains, please see: Registering domains - Amazon Route 53

Deploy with HTTPS support

To setup your ECS cluster based on a domain, please use the following CloudFormation parameters:

Domain
Your HTTPS domain name. For example: example.org
Email
An email for LetsEncrypt renewal alerts.

For example:

aws cloudformation create-stack --region=us-west-2 --stack-name earshot-stack --template-body file://templates/cloudformation-template.yaml --parameters ParameterKey=InstanceType,ParameterValue=t3.micro ParameterKey=KeyName,ParameterValue=<YOUR_KEY_PAIR_NAME> ParameterKey=RtmpAuthToken,ParameterValue=<YOUR_CUSTOM_AUTH_TOKEN> ParameterKey=FfmpegFlags,ParameterValue="-loglevel repeat+level+verbose" ParameterKey=Domain,ParameterValue=<YOUR_DOMAIN_NAME> ParameterKey=Email,ParameterValue=<YOUR_EMAIL> --capabilities CAPABILITY_IAM

Setup Elastic IP

If you plan to use Earshot with HTTPS support you will also need to create a Elastic IP. You can create a new one in the AWS console and associate it with your ECS instance.

To read more about EIPs please view: Elastic IP addresses - Amazon Elastic Compute Cloud

To find the name of your Earshot instance:

  1. Open AWS Console -> EC2
  2. In the left pane, click Instances
  3. Find the name of your EC2 instance

    note: Your instance name should begin with EC2ContainerService-. For example if you used Earshot as the stack name you would have a name such as EC2ContainerService-Earshot-EcsCluster-BzS7LfpNkDQq.

  4. Copy the name of your Earshot instance

Create new EIP:

You can create a new EIP using AWS console or CLI. Below are the steps needed to create a new EIP and associate it with your Earshot cluster:

  1. Allocate a new EIP.

    note: If you are new to AWS you can Learn more here: Elastic IP addresses - Amazon Elastic Compute Cloud

  2. In the Elastic IP console, click the checkbox next to your new EIP
  3. Click Actions -> Associate Elastic IP address
  4. In the instance field, please use the name of your Earshot instance
  5. Select any available private IP address
  6. Click Associate

Deploy Route53 stack

To deploy the Route53 stack, please run the following command:

aws cloudformation create-stack --region=us-west-2 --stack-name earshot-stack-dns --template-body file://templates/route53.yaml --parameters ParameterKey=ZoneName,ParameterValue=<Your Zone Name> ParameterKey=ZoneId,ParameterValue=<Your Zone ID> ParameterKey=ElasticIP,ParameterValue=<Your EIP>  --capabilities CAPABILITY_IAM