Question about custom VideoSink implementation

[roman-mityukov]

Hello, I need to implement video frame processing from a remote participant video in my application.

If I understand correct, I need to do something like this

                           class CustomVideoRenderer : VideoSink {
                                override fun onFrame(p0: VideoFrame?) {
                                    logd("onFrame $p0")
                                }
                            }

                            val videoRenderer = CustomVideoRenderer()

                            val room = LiveKit.create(
                                appContext = applicationContext,
                                options = RoomOptions(adaptiveStream = true, dynacast = true)
                            )

                            room.connect(
                                url = "someUrl",
                                token = "someToken"
                            )

                            room.events.collect { event ->
                                when (event) {
                                    is RoomEvent.TrackSubscribed -> {
                                        val track = event.track
                                        if (track is RemoteVideoTrack) {
                                            track.addRenderer(videoRenderer)
                                        }
                                    }

                                    else -> {}
                                }
                            }

but CustomVideoRenderer::onFrame is not called

Can you provide example of VideoSink implementation and usage please?

[roman-mityukov]

The VideoSink started working after replacing options with RoomOptions(adaptiveStream = false, dynacast = false)

[davidliu]

Ah, you'll want to implement VideoSinkVisibility to take advantage of adaptive streaming. Otherwise it'll receive the highest video quality regardless of view size, taking up more bandwidth and battery usage than may be needed.

I'll update the docs to be more clear about this.