element-hq / element-x-ios

Next generation Matrix client for iOS built with SwiftUI on top of matrix-rust-sdk.
https://element.io/labs/element-x
GNU Affero General Public License v3.0
420 stars 99 forks source link

[Element Call] Call Kit controls on the lock screen #3372

Open stefanceriu opened 3 weeks ago

stefanceriu commented 3 weeks ago

Ideally starting and Element Call should integrate with CallKit and use the following behaviors:

This was mostly working previously but, unfortunately, it changed with iOS 18 and ElementCall audio is no longer redirecting to the speaker, making the system believe the other behaviors are available e.g. ending the call when locking the screen.

The only solution we found for this to start working again was to correctly setup a CallKit session, something we weren't able to do previouly because of https://github.com/element-hq/element-x-ios/issues/3041 and the workarounds needed for that.

As such, the only option we have at our disposal at the moment is to remove the lock screen controls entirely and revisit them after we find out why the web side can't handle CallKit media sessions properly.

ara4n commented 2 weeks ago

@stefanceriu what happens if you use a trivial widget that runs getUserMedia() with callkit enabled media, rather than full element call? In other words, is this going wrong at the webrtc or livekit layer?

fkwp commented 1 week ago

@stefanceriu if its low hanging we can test with https://webrtc.github.io/samples/src/content/devices/input-output/

stefanceriu commented 1 week ago

@stefanceriu if its low hanging we can test with https://webrtc.github.io/samples/src/content/devices/input-output/

Nice one, yeah, this one has the same exact problems. Unfortunately though it doesn't print out anything in the console as to why.

ara4n commented 1 week ago

well, arse. does it fail on all possible AV settings? (e.g. is there any chance this is a MediaTrackConstraint problem?)