Metal API Validation - Video Call

[raphaelalkamim]

Describe the bug When using the app in Debug mode on the macOS (Designed for iPad) simulator, if doing video calls, we get an error and the app crashes. This error can be avoided by deactivating METAL API Validation in the Run app Build Diagnostics; however, this doesn't seem like a good idea. Could you please investigate what might be causing this bug?

SDK Version 2.0.5

iOS/macOS Version MacOS 14.5

Steps to Reproduce Start a video call on MacOS using designed for iPad target.

Expected behavior It should not be necessary to deactivate Metal API Validation to do video calls in Debug mode on simulators.

Screenshots N/A

Logs

2024-05-27T09:16:53+0100 info LiveKitSDK : [LiveKit] CameraCapturer.init(delegate:options:) isMultitaskingAccessSupported: false 2024-05-27T09:16:53+0100 info LiveKitSDK : [LiveKit] LocalParticipant._publish(track:options:) [publish] LocalVideoTrack(sid: nil, name: camera, source: Source(rawValue: 1)) options: nil... [C:1-4] Error received: Invalidated by remote connection. CMIO_DAL_CMIOExtension_Stream.mm:1472:GetPropertyData wrong data size for kCMIOStreamPropertyCenterStageFramingMode CMIOHardware.cpp:331:CMIOObjectGetPropertyData Error: 561211770, failed Fig assert: "err == 0 " at bail (CMIOUtilities.h:133) - (err=561211770) CMIO_DAL_CMIOExtension_Stream.mm:1472:GetPropertyData wrong data size for kCMIOStreamPropertyCenterStageFramingMode CMIOHardware.cpp:331:CMIOObjectGetPropertyData Error: 561211770, failed Fig assert: "err == 0 " at bail (CMIOUtilities.h:133) - (err=561211770) 2024-05-27T09:16:53+0100 info LiveKitSDK : [LiveKit] CameraCapturer.startCapture() starting camera capturer device: <AVCaptureFigVideoDevice: 0x135867a00 [FaceTime HD Camera][3F45E80A-0176-46F7-B185-BB9E2C0E82E3]>, format: (format: 'vide'/'420v' 1280x 720, {15- 30 fps}, photo dims:{1280x720}, max zoom:45.00 (upscales @1.00), ISO:50.0-50.0, supports multicam, supports Portrait Effect, supports Studio Lighting, supports Reactions, dimensions: Dimensions(1280x720)), fps: 30(0...30) <<<< FigCaptureCameraParameters >>>> Fig assert: "success" at bail (FigCaptureCameraParameters.m:250) - (err=0) <<<< FigCaptureCameraParameters >>>> Fig assert: "success" at bail (FigCaptureCameraParameters.m:250) - (err=0) |_ fsbpAutofocus | Fig assert: "err == noErr" at bail (FigSampleBufferProcessor_Autofocus.m:2484) - (err=0) -[MTLDebugDevice newTextureWithDescriptor:iosurface:plane:]:2641: failed assertion `Texture Descriptor Validation IOSurface textures must use MTLStorageModeShared

[hiroshihorie]

This is probably fixed, can you try main branch ?

[raphaelalkamim]

Hello, I have tested the main branch and am still experiencing the same behavior.

[davidzhao]

@raphaelalkamim do you have a way for us to reproduce this issue? Are you able to repro with the example app?

If not, could you attach a example project where we could repro?

[hiroshihorie]

Ok I can confirm this, Investigating... "Mac Catalyst" > OK "Designed for iPad" > Crashes.

[hiroshihorie]

This looks like a bug in Apple's Portrait mode related code. I don't think we can fix this at the moment. The workaround is to disable Metal validation...

I tried modifying code in our Metal renderer but it has nothing to do with it. https://github.com/webrtc-sdk/webrtc/compare/livekit-prefixed-m125...hiroshi/attempt-fix-metal-designed-for-ipad?expand=1

https://forums.developer.apple.com/forums/thread/710843