search

livekit / client-sdk-android

LiveKit SDK for Android
https://docs.livekit.io
Apache License 2.0
174 stars 69 forks source link

Ability to force video codec #475

Closed serhiynovos closed 3 weeks ago

serhiynovos commented 4 weeks ago

LiveKit supports multiple codecs such as: VP8, VP9, AV1, H264. We have some devices especially based on Rockchip 3588 where some encoders/decoders are now working correctly with LiveKit. Only VP9 is working for example. Looks like WebRTC does not have proper support for their Media Process Platform https://github.com/rockchip-linux/mpp .

To solve this problem already modified SimulcastVideoEncoderFactoryWrapper::supportedCodecs if it's RK3588 device, for others it was left by default to return all supported codecs.

native.supportedCodecs.filter { it.name.equals("VP9", ignoreCase = true) }
            .toTypedArray()

With latest version of SDK I noticed that if I do call from device to device and one device uses VP9 and another VP8, video from VP8 device is not visible on VP9 device. To solve it I had to go to LiveKit server settings and leave only VP9 codec available so both devices will use it and it will work fine. This approach creates some issues. I cannot use other codecs like h264 or vp8 with devices which support it. For example get h264 rtsp stream from camera and send to the room or use http url ingress to send h264 video to some group.

I think having ability to force codec will help to solve this problem.

davidliu commented 4 weeks ago

You can select your preferred codec with setting VideoTrackPublishOptions.videoCodec (available codecs can be found under the VideoCodec enum) when publishing the track. For the setCamera/ScreenShareEnabled methods where we handle the track creation and publication, you can change the Room.videoTrackCaptureDefaults.videoCodec instead.

serhiynovos commented 4 weeks ago

Thank you. Let me try it

serhiynovos commented 4 weeks ago

@davidliu Thanks. It worked fine. Do you have any ideas why some codecs may not work. Device either hangs completely or we see black stream

davidliu commented 4 weeks ago

Not sure, VP8 should be supported on all android devices, but I guess that's a hardware issue if it doesn't. You can try forcing software codecs for VP8 and see if that works (see CustomVideoEncoderFactory.forceSWCodecs).

serhiynovos commented 4 weeks ago

forceSWCodecs helped for VP8 but did not for h264, Still don't see any video track for h264. Also chrome://webrtc-internals does not show any inbound tracks

davidliu commented 3 weeks ago

Closing this since it seems like the issue is something else. H264 doesn't have a software encoder unfortunately, so there's no fallback we can use if the device doesn't support H264 encoding.