livekit / client-sdk-android

LiveKit SDK for Android
https://docs.livekit.io
Apache License 2.0
160 stars 63 forks source link

Sharing the screen, the other participant cannot see the full screen. #416

Closed DDPlay123 closed 1 month ago

DDPlay123 commented 1 month ago

Describe the bug When screen sharing, the recipient receives a partially cropped video and cannot see the entire shared content.

Expected behavior Ideally, the recipient should be able to see the entire screen sharing display.

Screenshots This is device A, and device A is responsible for sharing the screen.

This is device B, and device B displays the screen of device A.

Device Info:

Additional context This is my setup.

val room = LiveKit.create(
    appContext = context,
    options = RoomOptions(
        adaptiveStream = true,
        dynacast = true
    )
)

room.connect(url = LIVE_KIT_URL, token = token)

private fun handleRemoteParticipant(
    remoteParticipant: Participant
) {
    val videoTrackPubFlow = remoteParticipant::videoTrackPublications.flow
        .map { remoteParticipant to it }
        .flatMapLatest { (participant, videoTracks) ->
            val trackPublication = participant.getTrackPublication(Track.Source.SCREEN_SHARE)
                ?: participant.getTrackPublication(Track.Source.CAMERA)
                ?: videoTracks.firstOrNull()?.first

            flowOf(trackPublication)
        }

    viewLifecycleOwner.lifecycleScope.launch {
        val videoTrackFlow = videoTrackPubFlow
            .flatMapLatestOrNull { pub -> pub::track.flow }

        launch {
            videoTrackFlow.collectLatest { videoTrack ->
                setupRemoteVideoTrack(videoTrack as? VideoTrack)
            }
        }

        launch {
            videoTrackPubFlow
                .flatMapLatestOrNull { pub -> pub::muted.flow }
                .collectLatest { viewModel.setShowRemoteRender(!(it ?: true)) }
        }
    }

    setupRemoteVideoTrack(getVideoTrack(remoteParticipant))
}

private fun setupRemoteVideoTrack(videoTrack: VideoTrack?) = with(viewModel) {
    if (remoteBoundVideoTrack == videoTrack) {
        return@with
    }
    remoteBoundVideoTrack?.removeRenderer(binding.remoteRender)
    remoteBoundVideoTrack = videoTrack
    videoTrack?.addRenderer(binding.remoteRender)
}
davidliu commented 1 month ago

Have you tried setScalingType?

DDPlay123 commented 1 month ago

Yes, I have tried SCALE_ASPECT_FIT, SCALE_ASPECT_FILL, and SCALE_ASPECT_BALANCED. They all displayed the same result.

In fact, if you use a screen shared from a mobile phone, the result will match the SCALE_ASPECT_FIT ratio. However, if you share the screen from a tablet or a computer, the displayed result will follow the SCALE_ASPECT_FILL ratio. No matter what parameter I set the ScaleType of the TextureViewRenderer to, it has no effect.

davidliu commented 1 month ago

Ah, I think the problem is how the scalingType works here. The scalingType affects how the renderer view gets measured, rather than changing how the video renders within its view.

Try wrapping the TextureViewRenderer within a FrameLayout set to the size you need, and then using a width/height of wrap_content on the TextureViewRenderer.

DDPlay123 commented 1 month ago

Oh, I didn't notice this issue. I tried setting the height of the TextureViewRenderer to wrap_content, and now the problem is solved. Thank you very much for your suggested solution.