main |
|
dev |
|
issues |
|
commits |
|
AlphaRTC is a fork of Google's WebRTC project using ML-based bandwidth estimation, delivered by the OpenNetLab team. By equipping WebRTC with a more accurate bandwidth estimator, our mission is to eventually increase the quality of transmission.
AlphaRTC replaces Google Congestion Control (GCC) with two customized congestion control interfaces, PyInfer and ONNXInfer. The PyInfer provides an opportunity to load external bandwidth estimator written by Python. The external bandwidth estimator could be based on ML framework, like PyTorch or TensorFlow, or a pure Python algorithm without any dependencies. And the ONNXInfer is an ML-powered bandwidth estimator, which takes in an ONNX model to make bandwidth estimation more accurate. ONNXInfer is proudly powered by Microsoft's ONNXRuntime.
If you are preparing a publication and need to introduce OpenNetLab or AlphaRTC, kindly consider citing the following paper:
@inproceedings{eo2022opennetlab,
title={Opennetlab: Open platform for rl-based congestion control for real-time communications},
author={Eo, Jeongyoon and Niu, Zhixiong and Cheng, Wenxue and Yan, Francis Y and Gao, Rui and Kardhashi, Jorina and Inglis, Scott and Revow, Michael and Chun, Byung-Gon and Cheng, Peng and Xiong, Yongqiang},
booktitle={Proceedings of the 6th Asia-Pacific Workshop on Networking},
pages={70--75},
year={2022}
}
We recommend you directly fetch the pre-provided Docker images from opennetlab.azurecr.io/alphartc
or Github release
docker pull opennetlab.azurecr.io/alphartc
docker image tag opennetlab.azurecr.io/alphartc alphartc
wget https://github.com/OpenNetLab/AlphaRTC/releases/latest/download/alphartc.tar.gz
docker load -i alphartc.tar.gz
Ubuntu 18.04 or 20.04 is the only officially supported distro at this moment. For other distros, you may be able to compile your own binary, or use our pre-provided Docker images.
To compile AlphaRTC, please refer to the following steps
Prerequisites
Make sure Docker is installed on your system and add user to docker group.
# Install Docker
curl -fsSL get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker ${USER}
Clone the code
git clone https://github.com/OpenNetLab/AlphaRTC.git
Build Docker images
cd AlphaRTC
make all
You should then be able to see two Docker images, alphartc
and alphartc-compile
using sudo docker images
If you don't want to use Docker, or have other reasons to compile from scratch (e.g., you want a native Windows build), you may use this method.
Note: all commands below work for both Linux (sh) and Windows (pwsh), unless otherwise specified
Grab essential tools
You may follow the guide here to obtain a copy of depot_tools
Clone the repo
git clone https://github.com/OpenNetLab/AlphaRTC.git
Sync the dependencies
cd AlphaRTC
gclient sync
mv src/* .
Generate build rules
Windows users: Please use x64 Native Tools Command Prompt for VS2017. The clang version comes with the project is 9.0.0, hence incompatible with VS2019. In addition, environmental variable DEPOT_TOOLS_WIN_TOOLSCHAIN
has to be set to 0
and GYP_MSVS_VERSION
has to be set to 2017
.
gn gen out/Default
Compile
ninja -C out/Default peerconnection_serverless
For Windows users, we also provide a GUI version. You may compile it via
ninja -C out/Default peerconnection_serverless_win_gui
AlphaRTC consists of many different components. peerconnection_serverless
is an application for demo purposes that comes with AlphaRTC. It establishes RTC communication with another peer without the need of a server.
In order to run the application, you will need a configuration file in json format. The details are explained in the next chapter.
In addition to the config file, you will also need other files, such as video/audio source files and an ONNX model.
To run an AlphaRTC instance, put the config files in a directory, e.g., config_files
, then mount it to an endpoint inside alphartc
container
sudo docker run -v config_files:/app/config_files alphartc peerconnection_serverless /app/config_files/config.json
Since peerconnection_serverless
needs two peers, you may spawn two instances (a receiver and a sender) in the same network and make them talk to each other. For more information on Docker networking, check Docker Networking
This section describes required fields for the json configuration file.
serverless_connection
true
, the client will act as sender and automatically connect to receiver when launchedtrue
, the client will act as receiver and wait for sender to connect.Note: one and only one of sender.enabled
and receiver.enabled
has to be true
. I.e., sender.enabled
XOR receiver.enabled
bwe_feedback_duration: The duration the receiver sends its estimated target rate every time(in millisecond)
video_source
true
, the client will not take any video source as inputtrue
, then the client will use the web camera as the video source. For Linux, please set to false
true
, then the client will use a video file as the video sourcetrue
, the client will write log to the file specifiedNote: one and only one of video_source.webcam.enabled
and video_source.video_file.enabled
has to be true
. I.e., video_source.webcam.enabled
XOR video_source.video_file.enabled
audio_source
save_to_file
The default bandwidth estimator is PyInfer, You should implement your Python class named Estimator
with required methods report_states
and get_estimated_bandwidth
in Python file BandwidthEstimator.py
and put this file in your workspace.
There is an example of Estimator with fixed estimated bandwidth 1Mbps. Here is an example BandwidthEstimator.py.
class Estimator(object):
def report_states(self, stats: dict):
'''
stats is a dict with the following items
{
"send_time_ms": uint,
"arrival_time_ms": uint,
"payload_type": int,
"sequence_number": uint,
"ssrc": int,
"padding_length": uint,
"header_length": uint,
"payload_size": uint
}
'''
pass
def get_estimated_bandwidth(self)->int:
return int(1e6) # 1Mbps
If you want to use the ONNXInfer as the bandwidth estimator, you should specify the path of onnx model in the config file. Here is an example configuration receiver.json
Dockerized environment
To better demonstrate the usage of peerconnection_serverless, we provide an all-inclusive corpus in examples/peerconnection/serverless/corpus
. You can use the following commands to execute a tiny example. After these commands terminates, you will get outvideo.yuv
and outaudio.wav
.
PyInfer:
sudo docker run -d --rm -v `pwd`/examples/peerconnection/serverless/corpus:/app -w /app --name alphartc alphartc peerconnection_serverless receiver_pyinfer.json
sudo docker exec alphartc peerconnection_serverless sender_pyinfer.json
ONNXInfer:
sudo docker run -d --rm -v `pwd`/examples/peerconnection/serverless/corpus:/app -w /app --name alphartc alphartc peerconnection_serverless receiver.json
sudo docker exec alphartc peerconnection_serverless sender.json
Bare metal
If you compiled your own binary, you can also run it on your bare-metal machine.
Linux users:
Copy the provided corpus to a new directory
cp -r examples/peerconnection/serverless/corpus/* /path/to/your/runtime
Copy the essential dynanmic libraries and add them to searching directory
cp modules/third_party/onnxinfer/lib/*.so /path/to/your/dll
export LD_LIBRARY_PATH=/path/to/your/dll:$LD_LIBRARY_PATH
Start the receiver and the sender
cd /path/to/your/runtime
/path/to/alphartc/out/Default/peerconnection ./receiver.json
/path/to/alphartc/out/Default/peerconnection ./sender.json
Windows users:
Copy the provided corpus to a new directory
cp -Recursive examples/peerconnection/serverless/corpus/* /path/to/your/runtime
Copy the essential dynanmic libraries and add them to searching directory
cp modules/third_party/onnxinfer/bin/*.dll /path/to/your/dll
set PATH=/path/to/your/dll;%PATH%
Start the receiver and the sender
cd /path/to/your/runtime
/path/to/alphartc/out/Default/peerconnection ./receiver.json
/path/to/alphartc/out/Default/peerconnection ./sender.json
The OpenNetLab is an open-networking research community. Our members are from Microsoft Research Asia, Tsinghua Univeristy, Peking University, Nanjing University, KAIST, Seoul National University, National University of Singapore, SUSTech, Shanghai Jiaotong Univerisity.
You can find the Readme of the original WebRTC project here