search

awslabs / amazon-kinesis-video-streams-webrtc-sdk-c

Amazon Kinesis Video Streams Webrtc SDK is for developers to install and customize realtime communication between devices and enable secure streaming of video, audio to Kinesis Video Streams.
https://awslabs.github.io/amazon-kinesis-video-streams-webrtc-sdk-c/group__PublicMemberFunctions.html
Apache License 2.0
1.02k stars 307 forks source link

[QUESTION] #1065

Closed wingengineer closed 3 years ago

wingengineer commented 3 years ago

About adding support to PCM audio

I need PCM codec to verify my hardware (my hardware output PCM). Any suggestion for me to add PCM support ? e.g. modify based on the existing G711 or opus codec?

Thanks Wing

hassanctech commented 3 years ago

Can you please explain your set up. Are you using the WebRTC C SDK on the hardware from which you would like to output PCM? Is that the master or viewer side? If I assume it is the master side then I presume in the "answer" you would like to state that you support PCM? If I understood this correctly, please see the following section of code that you will need to change from OPUS to PCM (a-law or mu-law depending on what you want) see supported values here: https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c/blob/cd002e5d1b09dc4175aa3ca5339fd16cec4b85fa/src/include/com/amazonaws/kinesis/video/webrtcclient/Include.h#L701-L707

https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c/blob/master/samples/Common.c#L446 https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c/blob/master/samples/Common.c#L460

The side that creates the offer will also need to state that it supports this codec so that it can be selected for the actual streaming session.

Please let me know if I did not answer your question or if anything is unclear.

wingengineer commented 3 years ago

Hi, Thanks a lot, and Yes,You exactly know what's my situation.

  1. running WebRTC C-SDK on an embedded Linux,
  2. The MCU output PCM (but not a-law or u-ulaw)
  3. The MCU(embedded Linux) with and camera, and run as master side.

Since a-law and u-law audio quality is a bit low, I want to verify my hardware and the hardware (audio chips) only output PCM (raw), so can I "change" (modify) the SDK to support raw PCM?

Actually, I may try to also test AAC, so same question, can I port AAC support to webRTC? I ask this because I have a quick scan at the webrtc source code, it seem it only "prepare " the payload and send out, so it depend on the "viewer side" to decode, right? If the "viewer side " is Android App (also bsaed on the WebRTC SDK-Android), then I can easily add other codec support to the WebRTC C-SDK (from Master side) ??

Thanks again for your kindly replied.

hassanctech commented 3 years ago

Yes I think you looked here: https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c/tree/master/src/source/Rtp/Codecs

You're right if master side is sendonly it depends on the viewer side to decode so as long as the viewer side can decode and you can add codec support (decoding) on viewer side and encoding on master side then what you describe should work. But out of the box we only provide support for the codecs I specified.

You can feel free to add AAC support to WebRTC, we welcome contributions! One thing to note though is as far as I know browsers such as Firefox and Chrome do not support AAC over WebRTC. Our Android WebRTC SDK uses Google's WebRTC implementation (https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-android/blob/master/build.gradle#L125), so I do not think it will be trivial to add support there.

hassanctech commented 3 years ago

I am resolving this issue since the question is answered. If you have a new question please open a new issue.