Peer-to-peer streaming between the embedded SDK master and an Android device viewer: Can only view the video of the android device.

[rach21nag]

Master: Raspberry Pi with amazon kinesis video streams webrtc sdk c 1.7.2 Viewer: Pixel 4 API 30 Android 11.0 (Emulator) I can hear the audio faintly from the KVSWebrtcClientMaster. But cannot see the video

Raspberry Pi and kvs webrtc test page work perfectly with video from both the devices. Android device and kvs webrtc test page also work.

Please let me know what should be done.

[nickhuangcyh]

Hi @rach21nag,

I have the same problem as you and I finally solved this problem

Master: Linux embeded system with mediatek chip Viewer: Multiple Android Phones

The problem is that our embeded system only supports H264 video codec format but not VP8/VP9 video codec format Most Android devices only support VP8/VP9 video codec on WebRTC Currently WebRTC hardware encoding only supports devices that using QCOM and EXYNOS chips You can find out in the link below Why WebRTC only support H264 in Chrome but not in native application with some devices

You can check the SDP packet to find the supported video codec [For example] (Embedded System -> Android Phone SDP) m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 127 a=rtpmap:0 H264 ... (Android Phone -> Embedded System SDP) m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 127 a=rtpmap:96 VP8/90000

The supported video codecs on both sides do not match

And I can't change the embedded system chip to support VP8/VP9 codec So I can only solve this problem from Android's WebRTC SDK I modified and recompiled the source code of WebRTC so that it can also support H264 on different chips (AMD, Mediatek...)

1. Enable the hardware encoder for all chips
2. Enable the software encoder for all chips

And now it works perfectly on all my android devices

BTW I'm not a native english speaker, hope you can understand what i mean

BR, Nick

[rach21nag]

Thank you for your reply! I will try this out :)

[nickhuangcyh]

Hi @rach21nag,

Here is the file where i modify the source code of webrtc for your reference

1. Go to webrtc.github.io (Android development is only supported on Linux.)
2. Pull source code

3. modify these files below (Enable hardware video ecoder for all chips and software openh264 media codec)

   diff --git a/sdk/android/api/org/webrtc/HardwareVideoEncoderFactory.java b/sdk/android/api/org/webrtc/HardwareVideoEncoderFactory.java
   index 9cd4aa2d37..9903300a36 100644
   --- a/sdk/android/api/org/webrtc/HardwareVideoEncoderFactory.java
   +++ b/sdk/android/api/org/webrtc/HardwareVideoEncoderFactory.java
   @@ -232,11 +232,15 @@ public class HardwareVideoEncoderFactory implements VideoEncoderFactory {
      return false;
    }
    String name = info.getName();
   +
   +    return true;
   +    /*
    // QCOM H264 encoder is supported in KITKAT or later.
    return (name.startsWith(QCOM_PREFIX) && Build.VERSION.SDK_INT >= Build.VERSION_CODES.KITKAT)
        // Exynos H264 encoder is supported in LOLLIPOP or later.
        || (name.startsWith(EXYNOS_PREFIX)
               && Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP);
   +    */
   }
   
   private boolean isMediaCodecAllowed(MediaCodecInfo info) {
   diff --git a/sdk/android/src/java/org/webrtc/MediaCodecUtils.java b/sdk/android/src/java/org/webrtc/MediaCodecUtils.java
   index 5d83014dc3..bb7bed4e29 100644
   --- a/sdk/android/src/java/org/webrtc/MediaCodecUtils.java
   +++ b/sdk/android/src/java/org/webrtc/MediaCodecUtils.java
   @@ -30,7 +30,7 @@ class MediaCodecUtils {
   static final String NVIDIA_PREFIX = "OMX.Nvidia.";
   static final String QCOM_PREFIX = "OMX.qcom.";
   static final String[] SOFTWARE_IMPLEMENTATION_PREFIXES = {
   -      "OMX.google.", "OMX.SEC.", "c2.android"};
   +      "OMX.SEC.", "c2.android"};
   
   // NV12 color format supported by QCOM codec, but not declared in MediaCodec -
   // see /hardware/qcom/media/mm-core/inc/OMX_QCOMExtns.h

4. Build .aar for Android tools_webrtc/android/build_aar.py --arch "armeabi-v7a" "arm64-v8a" --extra-gn-args='rtc_use_h264=true ffmpeg_branding = "Chrome" proprietary_codecs=true'
5. 🎉