amazon-kinesis-video-streams-webrtc-sdk-c-for-freertos

This project demonstrate how to port Amazon Kinesis Video WebRTC C SDK to FreeRTOS. It uses the ESP-Wrover-Kit as a reference platform. You may follow the same procedure to port to other hardware platforms.

Clone projects

Please git clone this project using the command below. This will git sub-module all depended submodules under main/lib.

git submodule update --init --recursive

Reference platform

We use ESP IDF 4.4.2 and the ESP-Wrover-Kit as the reference platform.

Please git clone the ESP IDF 4.4.2 with following command

git clone -b v4.4.2 --recursive https://github.com/espressif/esp-idf.git esp-idf-v4.4.2

make change on file

components/esp_rom/include/esp32/rom/ets_sys.h

line 638 to 644 to following

+#ifndef STATUS typedef enum { OK = 0, FAIL, PENDING, BUSY, CANCEL, } STATUS; +#endif

Please follow the Espressif instructions to set up the environment.

ESP IDF 4.4.2 only supports Python version 3.10 and below and was tested on version 3.9.16.

Apply patches

Next, patch depended libraries for using with WebRTC.

wslay

This project uses wslay as the websocket client. Please apply patches located in patch/wslay directory.

main/lib/wslay$ git am ../../../patch/wslay/*

libsrtp

This project uses v2.3.0 of libsrtp. Please apply patches located in patch/libsrtp directory.

main/lib/libsrtp$ git am ../../../patch/libsrtp/*

usrsctp

Please apply patches as below.

main/lib/usrsctp$ git am ../../../patch/usrsctp/*

[amazon-kinesis-video-streams-webrtc-sdk-c]

Please apply patches as below.

main/lib/amazon-kinesis-video-streams-webrtc-sdk-c$ git am ../../../patch/amazon-kinesis-video-streams-webrtc-sdk-c/*

If you run into problems when "git am" patches, you can use the following commands to resolve the problem. Or try "git am --abort" the process of git am, then "git apply" individual patches sequentially (in the order of the sequence number indicated by the file name).

$git apply --reject ../../../patch/problem-lib/problems.patch
// fix *.rej
$git add .
$git am --continue

Configure the project

Use menuconfig of ESP IDF to configure the project.

idf.py menuconfig

• These parameters under Example Configuration Options must be set.

• • ESP_WIFI_SSID
  • ESP_WIFI_PASSWORD
  • ESP_MAXIMUM_RETRY
  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY
  • AWS_DEFAULT_REGION
  • AWS_KVS_CHANNEL
  • AWS_KVS_LOG_LEVEL

• The modifications needed by this project can be seen in the sdkconfig file located at the root directory.

Video source

This project uses pre-recorded h.264 frame files for video streaming. Please put the files on a SD card. The files should look like:

/sdcard/h264SampleFrames/frame-%04d.h264.

The “%04d” part of the file name should be replaced by a sequence number of the frame.

There are pre-generated video frame files under

main/lib/amazon-kinesis-video-streams-webrtc-sdk-c/samples/h264SampleFrames/

ready to be copied to sdcard.

Please note that you can not use J-TAG and SD card simultaneously on ESP-Wrover-Kit because they share some pins.

Generate video source

Given a video file videotestsrc, the following GStreamer command generates video frame files. If you want to reduce the number of video files, please modify related setting in sample code.

sh
gst-launch-1.0 videotestsrc pattern=ball num-buffers=1500 ! timeoverlay ! videoconvert ! video/x-raw,format=I420,width=1280,height=720,framerate=5/1 ! queue ! x264enc bframes=0 speed-preset=veryfast bitrate=128 byte-stream=TRUE tune=zerolatency ! video/x-h264,stream-format=byte-stream,alignment=au,profile=baseline ! multifilesink location="frame-%04d.h264" index=1

Build and Flash

Build the project and flash it to the board, then run monitor tool to view serial output:

idf.py -p PORT flash monitor

(To exit the serial monitor, type Ctrl-].)

See the Getting Started Guide of ESP IDF for full steps to configure and use ESP-IDF to build projects.

Known limitations and issues

This project does not use audio at this point of time. When running on the ESP-Wrover-Kit, this project can only run at low frame rate and low bit rate.

The current implementation does not support data channel. Please check back later for availability of the data channel feature.

The m-line mismatch

When using the WebRTC SDK Test Page to validate the demo, you may get m-line mismatch errors. Different browsers have different behaviors. To work around such errors, you need to run the sample in amazon-kinesis-video-streams-webrtc-sdk-js, and disable audio functionality of audio. This patch disables the audio functionality. A later release of this project may eliminate the need for this.

patch/amazon-kinesis-video-streams-webrtc-sdk-js/0001-diable-offerToReceiveAudio.patch`

Quickly steps to do above

1. Click https://awslabs.github.io/amazon-kinesis-video-streams-webrtc-sdk-js/examples/index.html

2. Right click on the page, click 'save as', make sure the format is 'Webpage, Complete" and save. The 'KVS WebRTC Test Page.html' file and 'KVS WebRTC Test Page_files' directory are save.

3. Enter the 'KVS WebRTC Test Page_files' directory and patch the 'viewer.js' file by commented out SDP offer to receive audio.

   console.log('[VIEWER] Creating SDP offer'); await viewer.peerConnection.setLocalDescription( await viewer.peerConnection.createOffer({ // offerToReceiveAudio: true, offerToReceiveVideo: true, }), );

4. Load the page

   KVS WebRTC Test Page.html

   for WebRTC testing

Security

See CONTRIBUTING for more information.

License

This project is licensed under the Apache-2.0 License.