Shared screen video is shown with a big zoom for receiver, after 3-10 sec shared screen video returns to a normal view on iOS SDK

[SelimEmre]

Regarding iOS SDK: When shared screen video is shown with a big zoom - so far from our side it has been observed mostly if the user who is sharing the screen has slow internet connection.

Scenario for reproduction:

1. Two Users Log into QTS SDP, user1 logs into QTS Mobile Test iOS application. user2 either from iOS application, either from Web. Both users join one and the same session
2. User1 taps on "Screen share" icon, and clicks "Record screen" -> Screen of user1 is shown on user2 screen

Environment: Develop, Master

Actual result: Shared screen video is shown with a big zoom for receiver, after 3-10 sec shared screen video returns to a normal view

Expected result: Shared screen video is shown without a big zoom for receiver

One of the devices from which this issue has been observed when screen was shared: iPad Model Number MRJN2HC/A (iOS 15.5)

[mekya]

Thank you for the issue @SelimEmre .

Is this issue reproducible in the sample project?

[shibupanda]

Same issue we are also facing.

[yashtandon113]

[mekya]

Shall we have the log files for the issues? @shibupanda

It will be great if we have stream id and datetime that help us look at the correct location in the log file.

[shibupanda]

@mekya

ant-media-server.log antmedia-error.log

streamId: audioStream1 Time: between 2022-08-19 10:13:06 to 2022-08-19 10:14:06

FYI: also I got the same in recorded mp4 that means issue with recorded steam too something wrong with server it seems.

[shibupanda]

Hi @mekya did you get anything in the logs

[mekya]

Hi @shibupanda ,

I try to schedule this issue for this week. Hopefully, we'll have something on thursday community hours.

Let's meet on community hours again.

You can get the invitation for community hours by clicking "Schedule Meeting" (Get a Free Consultancy) on antmedia.io

Thank you

[shibupanda]

For Information: Now it is so frequent be there for long time sometimes 1-2 mins

[mekya]

Hi @shibupanda ,

I'm working on that. I'll keep you updated.

[shibupanda]

Hi @shibupanda ,

I'm working on that. I'll keep you updated.

Thanks @mekya

[mekya]

Hi @shibupanda ,

Server log files don't give too much information for that. I've some other ideas.

1. Could you please record the file as MP4 and let's see if we have the same problem?
2. Could you just use the sample code that we've provided with iOS SDK and take a look at if you reproduce the problem?
3. Do you share the logs on the iOS side?

Please try these things and let me know. We can even schedule a quick call to try to understand the problem?

[shibupanda]

Could you please record the file as MP4 and let's see if we have the same problem? Yes getting same I updated in my above comment

Could you just use the sample code that we've provided with iOS SDK and take a look at if you reproduce the problem? Sure I will try again

Do you share the logs on the iOS side? What do you mean by it ?

[mekya]

Do you share the logs on the iOS side? What do you mean by it ?

Do you see any logs on XCode when this problem happens?

[shibupanda]

@mekya Xcode logs: antmedia.log Just now we have tried and at initially only it came. it came twice in this short session ( with sample SDK)

Could you just use the sample code that we've provided with iOS SDK and take a look at if you reproduce the problem?

Yes this is happening with sample SDK shared with us. SDK we got from support team as we got lots of error while building app from git.

Below changes we have done to make video enabled in SampleHandler File

`var videoEnabled: Bool = true; var audioEnabled: Bool = true;

//with default sdk videoEnabled = false; //Changed one videoEnabled = true;

//with default sdk self.client.setOptions(url: url as! String, streamId: audioStreamId as! String, token: token as? String ?? “”, mode: AntMediaClientMode.publish, enableDataChannel: true, captureScreenEnabled: false); //Changed one self.client.setOptions(url: url as! String, streamId: audioStreamId as! String, token: token as? String ?? “”, mode: AntMediaClientMode.publish, enableDataChannel: true, captureScreenEnabled: true);

//with default sdk if (videoEnabled != nil) { self.client.setVideoEnable(enable: videoEnabled); self.client.setExternalVideoCapture(externalVideoCapture: false); } //Changed one if (videoEnabled != nil) { self.client.setVideoEnable(enable: videoEnabled); self.client.setExternalVideoCapture(externalVideoCapture: true); }

`

[mekya]

Thank you. I think I understand the problem. I'm digging into this.

[mekya]

Hi @shibupanda,

I think the problem has been resolved. I've confirmed for myself.

Could you please try this branch on your end? I hope it will work for you as well.

https://github.com/ant-media/WebRTC-iOS-SDK/tree/fix-crop-in-broadcast-extension

[shibupanda]

Hi @mekya While trying to build from git source we are getting lots of compile error something like Start scream not found.

When we tried to add the changes in the SDK provided by support team we are getting same cropped screen.

Your changes related to scale it seems. I am ok connect with you to debug this issue.

[shibupanda]

My mistake it is working now after we built using export_xc_framework.sh command. Thanks will debug more if any other thing gets impacted. Thank you

[mekya]

Hi @shibupanda ,

Thank you for letting me know.

I think we need to make the documentation very clear.

Please let me know if it resolves the problem for you

Regards, A. Oguz

[shibupanda]

Yes it is resolved but screen getting zig Zag while swiping up.

[mekya]

Could you show(screenshot, video) how it's getting zig zag while swiping up?

[shibupanda]

https://github.com/shibupanda/stream_video/blob/main/8b8c9dd7-367c-4b12-b068-748dd798edee-2022-08-26_12-01-42.853.mp4?raw=true

@mekya please check the video.

[mekya]

Thank you. I think this is HLS playback?

[shibupanda]

Thank you. I think this is HLS playback?

Yes

[mekya]

Yes, you're right. I've a workaround solution for you.

1. You can play with WebRTC and there is no zig zag problem there
2. You can adaptive bitrate on the server to play with HLS. It also resolves zig zag problem.

For permanent solution: I've spent a couple of hours to fix this issue today. However I could not succeed. I think the problem is about the pixel format. If I can convert the incoming pixel format from kCVPixelFormatType_420YpCbCr8BiPlanarFullRange to kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange, it will likely be resolved.

Could you just proceed with workaround solutions for now?

[shibupanda]

Sorry @mekya my information was bit incorrect

You can play with WebRTC and there is no zig zag problem there

We are actually playing with WebRtc only and the video what I shared after enabling webrtc codec support h264.

And to enable abr in server side will come up with additional cost as this needs GPU machine.

[mekya]

OK. Thank you. Just for clarification. Then you're good to go for your project or do you have any questions or expectations from my side?

[shibupanda]

@mekya did you check the shared video that is in webrtc screen getting zig Zag while swiping up even it's in WebRtc.

[mekya]

I encounter zig zag problem in HLS playback. I don't get zip zag while swiping up in WebRTC.

H264 is enabled on the server side and there is no ABR in the settings

[shibupanda]

H264 is enabled on the server side and there is no ABR in the settings

Yes there is no ABR even just to verify I enabled ABR still the problem exist. we can help to show this if someone wants to debug.

[mekya]

OK. We can have a meeting in the week days.

One question, Do you have this problem after fixing the big zoom issue or did you have this problem in the older version as well?

[shibupanda]

One question, Do you have this problem after fixing the big zoom issue or did you have this problem in the older version as well?

Yes after the fixes but even in both the version recorded video quality is not satisfactory some red blue colour is coming while with recorded video. you can see in that video.

[shibupanda]

@mekya did you figured it out anything here?

[mekya]

Hi @shibupanda ,

I've some more tests and I confirm that this pixelaxing(zig-zag) problem happens in older way.

I undestand the problem why it's happening but I need some more time for this. So my plan is to create another issue and fix it there because it's a different problem.

Btw, I think this problem's(zig-zag) severity is lower than this cropping issue. I hope you can continue with the current state because I've some other commitments to fulfill.

We can discuss it in the community hours as well if you want. https://community.antmedia.io/topic/1363-community-hours-free-coffee-sep-1-at-300-pm-gmt3-be-part-of-the-journey/#comment-4916