Envelop Ambisonic RTMP Streaming Higher-Order Transcoder (Earshot) is a containerized multichannel RTMP->DASH transcoder, based on ngnix. Earshot can be used to transcode higher-order Ambisonics and other multichannel live streams for the web.
Envelop is a nonprofit organization that amplifies the connective power of music through immersive listening spaces and open source spatial audio software. Three-dimensional experiences of sound and music bring people together, catalyzing shared moments of inspiration, empathy and wonder.
Tools such as Envelop for Live have made it easy for musicians to compose and perform ambisonic content. However, there remain limited options for live streaming ambisonic content, particularly beyond first order. Ambisonic live streaming has applications for VR/AR/XR and immersive home listening experiences.
Earshot can be used in combination with pkviet's OBS Studio Music Edition which supports multichannel AAC encoding up to 16.0. For more information on OBS see obsproject.com.
Earshot is designed to be easily deployed to a cloud-based hosting solution, such as AWS ECS, DigitalOcean, etc.
Earshot is GPL licensed. However, if you are interested in a commercial license for Earshot, please contact us at envelop@envelop.us.
NOTE: Earshot contains binaries that are inflated with Git LFS. You must install Git LFS first, then git clone
this repository. Downloading Earshot as a zip file of any kind (master or a release) is not supported and will not work.
To generate a multichannel audio stream in a format such as third-order AMBIX you can use any of the following:
tester/resources
directory` streamed with FFmpeg1. Create your environment settings
To setup your environment, first create a copy of the .env.example file. For example:
cp .env.example .env
Please update the environmental variables in the .env file as needed.
Note: if you are using SSL please be sure to add the domain and email fields you will be serving Earshot from. Also, if you will be running tests, please note that the tests will use "stream1" as the stream key. If you are using a custom authentication server (please see RTMP Authentication section for more on this) and will be using tests, please fill out
RTMP_AUTH_TOKEN
so that it holds the auth token for the stream key: "stream1".
2. Build and run the Docker container for the transcoder
From this (project root) directory:
docker-compose up --build nginx-rtmp
3. Open OBS Music Edition
Click Settings and set the following:
Stream
Output
Audio
4. Select Audio Source
tester/resources/16chambixloop.wav
file with Loop checked. You should now see audio under the Media Source in the Input Mixer.5. Start Streaming
Click Start Streaming in OBS
Your DASH stream manifest is now available at http://localhost/dash/stream1.mpd
DASH stream webtools are available under http://localhost/webtools
To add a RTMP auth secret token you can update the "RTMP_AUTH_TOKEN" environment variable in the docker-compose.yml or .env file, e.g. - RTMP_AUTH_TOKEN=my_secret
.
In order to prevent brute force attacks, Earshot will limit authentication requests to one per second. To allow for flexibility and legitimate traffic surges, occasional bursts for up to two requests are accepted, until, on avarage, the one per second limit is retained.
To use a custom http auth server, you can update the RTMP_AUTH_URL
environment variable. Earshot will pass name
(the Stream Key), token
(the auth token provided by the user), and other parameters to your server. You can read the full list of parameters.
On your streaming client, appent the secret using the token
GET parameter to the request.
ffmpeg -y -stream_loop -1 -i tester/resources/16chambixloop.wav -af "channelmap=channel_layout=hexadecagonal" -c:a aac -ac 16 -b:a 2048k -f flv "rtmp://127.0.0.1:1935/live/stream1?token=my_secret"
?token=my_secret
. If the stream name is stream1, Stream Key should be stream1?token=my_secret
If you want to add additional flags for ffmpeg that is called within the transcoder -- for example, more adaptive DASH stream bitrates -- you can update the "FFMPEG_FLAGS" environment variable in the docker-compose.yml.
open() "/usr/local/nginx/html/stream1.mpd" failed (2: No such file or directory)
errors when streaming.You may be missing key binaries installed by Git LFS. Make sure you have used git clone
to get Earshot, not downloaded it a zip file. If you installed Git LFS after cloning this repository, you can run git lfs checkout
to inflate the binaries.
You can look at the nginx-rtmp ffmpeg logs in the container. First open a shell:
docker exec -it earshot_nginx-rtmp_1 sh
Then tail the log file here:
tail -f /tmp/nginx_rtmp_ffmpeg_log
Load http://localhost/webtools in your browser to monitor and debugs streams from the transcoder. Chrome or Firefox preferred.
To test different Dash.js Player settings, edit the JSON in the Dash.js settings box. Any changes applied will affect the live stream on your browser in real time, or reload the new URL -- containing the encoded settings as a parameter -- to test the player settings loaded initially on page load.
Known Webtools issues:
Earshot can be easily deployed to AWS using AWS CLI and CloudFormation. You can customize the deployment configuration in the CloudFormation template file in the templates/
directory
A sample configuration for adaptive bandwidth streaming can be found in the cloudformation-template-adaptive.yaml
file with a c5n.2xlarge EC2 instance type. With an 1080p input stream with additional 720p and 360p adaptive resolutions and 16 channels at 128k each, CPU usage was found to be at 33% with no clients for this instance type.
Before deploying, you will first need to generate an EC2 Key Pair.
aws cloudformation create-stack --region=us-west-2 --stack-name earshot-stack --template-body file://templates/cloudformation-template.yaml --parameters ParameterKey=InstanceType,ParameterValue=t3.micro ParameterKey=KeyName,ParameterValue=<YOUR_KEY_PAIR_NAME> --capabilities CAPABILITY_IAM
To get the public URL of your Earshot instance:
This is the stream URL you should use for OBS or whichever live streaming application you are using at the source, e.g.: rtmp://<YOUR_INSTANCE_PUBLIC_DNS>:1935/live/stream1
To access Webtools, navigate to: http://<YOUR_INSTANCE_PUBLIC_DNS>/webtools
The ECS container currently uses predefined values for RTMP authentication and FFmpeg flags.
If you want to change the container environment variables you can change the stack variables when you deploy it. Or if you want to make changes live, you can update the task definition environment in the AWS console.
RtmpAuthToken
This is your RTMP authentication token
FfmpegFlags
Add additional flags to FFMPEG
aws cloudformation create-stack --region=us-west-2 --stack-name earshot-stack --template-body file://templates/cloudformation-template.yaml --parameters ParameterKey=InstanceType,ParameterValue=t3.micro ParameterKey=KeyName,ParameterValue=<YOUR_KEY_PAIR_NAME> ParameterKey=RtmpAuthToken,ParameterValue=<YOUR_CUSTOM_AUTH_TOKEN> ParameterKey=FfmpegFlags,ParameterValue="-loglevel repeat+level+verbose" --capabilities CAPABILITY_IAM
Google Chrome, amongst other browers, requires HTTPS to serve MPEG DASH streams.
One way to achieve this in a production environment is to wrap Earshot in a CDN like Amazon Cloudfront. Cloudfront will serve requests over HTTP and can interface with the Earshot server via plain old HTTP (the "origin"). You can read more on how to achieve this with CloudFront [here].(https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/live-streaming.html)
feature/ssl
branch.Adding a domain to your earshot service is an optional step. However, there are many benefits to it, such as HTTPS support and more.
To use the Route53 Cloudformation stack, you must have already purchased a domain from AWS.
For more info on registering Route53 domains, please see: Registering domains - Amazon Route 53
To setup your ECS cluster based on a domain, please use the following CloudFormation parameters:
Domain
Your HTTPS domain name. For example: example.org
Email
An email for LetsEncrypt renewal alerts.
For example:
aws cloudformation create-stack --region=us-west-2 --stack-name earshot-stack --template-body file://templates/cloudformation-template.yaml --parameters ParameterKey=InstanceType,ParameterValue=t3.micro ParameterKey=KeyName,ParameterValue=<YOUR_KEY_PAIR_NAME> ParameterKey=RtmpAuthToken,ParameterValue=<YOUR_CUSTOM_AUTH_TOKEN> ParameterKey=FfmpegFlags,ParameterValue="-loglevel repeat+level+verbose" ParameterKey=Domain,ParameterValue=<YOUR_DOMAIN_NAME> ParameterKey=Email,ParameterValue=<YOUR_EMAIL> --capabilities CAPABILITY_IAM
If you plan to use Earshot with HTTPS support you will also need to create a Elastic IP. You can create a new one in the AWS console and associate it with your ECS instance.
To read more about EIPs please view: Elastic IP addresses - Amazon Elastic Compute Cloud
To find the name of your Earshot instance:
note: Your instance name should begin with EC2ContainerService-
. For example if you used Earshot as the stack name you would have a name such as EC2ContainerService-Earshot-EcsCluster-BzS7LfpNkDQq.
You can create a new EIP using AWS console or CLI. Below are the steps needed to create a new EIP and associate it with your Earshot cluster:
note: If you are new to AWS you can Learn more here: Elastic IP addresses - Amazon Elastic Compute Cloud
To deploy the Route53 stack, please run the following command:
aws cloudformation create-stack --region=us-west-2 --stack-name earshot-stack-dns --template-body file://templates/route53.yaml --parameters ParameterKey=ZoneName,ParameterValue=<Your Zone Name> ParameterKey=ZoneId,ParameterValue=<Your Zone ID> ParameterKey=ElasticIP,ParameterValue=<Your EIP> --capabilities CAPABILITY_IAM