Video Quality is bad at receiver end during the screen sharing using ReplayKit in iOS

[sauravkure]

Description

Video Quality is bad at receiver end during the screen sharing using ReplayKit in iOS I am using ReplayKit for sharing screen. I have created broadcast extension in which I have added "ReplayKitVideoSource.swift" and "SampleHandler.swift" file. Integrated screen sharing feature in our app properly but at receiver end video quality is not so good. I need to improve quality of video during screen sharing. Please help me to find out the solution of this issue.

Steps to Reproduce

1. Create a Video room from iPhone and add 3 participant in this room (one from iOS, one from android, one from web)
2. Share screen from iPhone
3. At all receiver end video quality is not so good (distorted video is showing at receiver end)

Code

// Code that helps reproduce the issue

private func connectToRoom(_ pAccessToken : String? = nil)
    {
        print("SampleHandle-Debug \(#function)")
        //TwilioVideoSDK.audioDevice = self.audioDevice
        guard let lAccessToken = pAccessToken, !lAccessToken.isEmpty else {
            return
        }
        self.accessToken = lAccessToken
        //accessToken="tset"

        // This source will attempt to produce smaller buffers with fluid motion.
        let options = ReplayKitVideoSource.TelecineOptions.disabled //p30to24or25
        let (encodingParams, outputFormat) = ReplayKitVideoSource.getParametersForUseCase(codec: SampleHandler.kVideoCodec,
                                isScreencast: false,
                                telecineOptions: options)

        videoSource = ReplayKitVideoSource(isScreencast: false, telecineOptions: options)
        screenTrack = LocalVideoTrack(source: videoSource!,
                                      enabled: true,
                                      name: SampleHandlerConstants.K_SCREEN)

        videoSource!.requestOutputFormat(outputFormat)
        //audioTrack = LocalAudioTrack()

        let connectOptions = ConnectOptions(token: accessToken) { (builder) in

            // Use the local media that we prepared earlier.
            //builder.audioTracks = [self.audioTrack!]
            builder.videoTracks = [self.screenTrack!]

            // We have observed that downscaling the input and using H.264 results in the lowest memory usage.
            builder.preferredVideoCodecs = [SampleHandler.kVideoCodec]

            /*
             * Constrain the bitrate to improve QoS for subscribers when simulcast is not used, and to reduce overall
             * bandwidth usage for the broadcaster.
             */
            builder.encodingParameters = encodingParams

            /*
             * A broadcast extension has no need to subscribe to Tracks, and connects as a publish-only
             * Participant. In a Group Room, this options saves memory and bandwidth since decoders and receivers are
             * no longer needed. Note that subscription events will not be raised for remote publications.
             */
            builder.isAutomaticSubscriptionEnabled = false

            // The name of the Room where the Client will attempt to connect to. Please note that if you pass an empty
            // Room `name`, the Client will create one for you. You can get the name or sid from any connected Room.

            if let lRoomName = SampleHandlerConstants.getContentFromUserDefault(SampleHandlerConstants.ROOM_NAME) as? String,
               !lRoomName.isEmpty
            {
                builder.roomName=lRoomName
            }
            //builder.uuid=UUID(uuidString:uid)

//            if #available(iOS 12.0, *) {
//                builder.roomName = "Broadcast"
//            } else {
//                builder.roomName = setupInfo?[SampleHandler.kBroadcastSetupInfoRoomNameKey] as? String
//            }
        }

        // Connect to the Room using the options we provided.
        room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)
    }

#### Expected Behavior

When we share screen from iPhone then video of  screen sharing at receiver  must have good quality.

#### Actual Behavior
When we share screen from iPhone then video of  screen sharing at receiver  showing distorted (video quality not so good). 

#### Reproduces How Often

All times

### Versions

All relevant version information for the issue.

#### Video iOS SDK

'TwilioVideo', '4.5.0'

#### Xcode

12.5

#### iOS Version

14.4

#### iOS Device

iPhone 11

[sauravkure]

Any update on this issue?

[manishmg]

Hi @sauravkure, hope you are well. Did you achieve screen broadcast functionality with twillio?

[sauravkure]

Yes, screen broadcast functionality implemented successfully in app but shared screen video on participant end is showing distorted.

[manishmg]

@sauravkure can you please provide a sample of the screen broadcast? In my case, we are connecting group calls from the main app, on the click of the broadcast picker button we are calling video broadcast from the extension, but for that, we have to rejoin the video call from the extension. Once I get connected from the extension my screen is shared with others but others' video is blank for me.

Could you please help?

[sauravkure]

Please check below code for connnecting shared video in Group call:

`override func broadcastStarted(withSetupInfo setupInfo: [String : NSObject]?) { processToConnectFromRoom() } private func processToConnectFromRoom() { SampleHandlerConstants.getTwilioAccessTokenForScreenShare { accessToken, error in guard let lAccessToken = accessToken, !lAccessToken.isEmpty else { print("ScreenShare-Debug broadcastStarted getTwilioAccessTokenForScreenShare = (String(describing: error)) ") return } self.connectToRoom(lAccessToken) } }

private func connectToRoom(_ pAccessToken : String? = nil)
{
    guard let lAccessToken = pAccessToken, !lAccessToken.isEmpty else {
        return
    }
    self.accessToken = lAccessToken
    let options = ReplayKitVideoSource.TelecineOptions.p30to24or25 5
    let (encodingParams, outputFormat) = ReplayKitVideoSource.getParametersForUseCase(codec: SampleHandler.kVideoCodec,
                            isScreencast: false,
                            telecineOptions: options)

    videoSource = ReplayKitVideoSource(isScreencast: false, telecineOptions: options)
    screenTrack = LocalVideoTrack(source: videoSource!,
                                  enabled: true,
                                  name: SampleHandlerConstants.K_SCREEN)
    videoSource!.requestOutputFormat(outputFormat)

    let connectOptions = ConnectOptions(token: accessToken) { (builder) in
        builder.videoTracks = [self.screenTrack!]

        builder.preferredVideoCodecs = [SampleHandler.kVideoCodec]

        builder.encodingParameters = encodingParams

        builder.isAutomaticSubscriptionEnabled = false

        builder.region = SampleHandlerConstants.K_REGION
        if let lRoomName = SampleHandlerConstants.getContentFromUserDefault(SampleHandlerConstants.ROOM_NAME) as? String,
           !lRoomName.isEmpty
        {
            builder.roomName=lRoomName
        }

    }
    room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)
}`

If you need more support then please ping me over s_k_y_p_e (sauravsinha1707)