search

daily-demos / party-line

multiplatform audio only demos
BSD 2-Clause "Simplified" License
59 stars 26 forks source link

iOS - Mute/unmute not working correctly #11

Closed aidan-waite closed 3 years ago

aidan-waite commented 3 years ago

Repro steps

  1. Mac: Clone repo
  2. Mac: Open xcodeproj
  3. Mac: Set code signing to my developer account
  4. Mac: Build and run on iPhone
  5. iPhone: Enter name and last name and click create/join
  6. iPhone: Copy join code
  7. Mac: Go to https://partyline.daily.co/ in Chrome
  8. Mac: Join the chat with the code
  9. I've setup my mac to play music through its audio input. Unmuting mic on my mac lets me hear the music coming through the call on my phone
  10. iOS: Tap unmute

Issue

Expected: I can still hear the audio from my Mac but now I'm unmuted. Unmuting should only affect my iPhone user's mic settings not the audio playback.

Actual: I can no longer hear the audio from the mac once I unmute. Tapping mute/unmute doesn't ever bring back the audio from the mac.

Notes

regexident commented 3 years ago

Thanks for the bug report @aidan-waite!

I did a quick code review myself and this is the method that generates the "mic" button:

https://github.com/daily-demos/party-line/blob/42fed09a9523ae5949da2d24d66f6e5a6eb076d1/ios/Party%20Line/Views/RoomView.swift#L85-L99

Which calls this method on "action":

https://github.com/daily-demos/party-line/blob/42fed09a9523ae5949da2d24d66f6e5a6eb076d1/ios/Party%20Line/Views/RoomView.swift#L184-L190

Which then in turn calls this method on the Client:

https://github.com/daily-demos/party-line/blob/42fed09a9523ae5949da2d24d66f6e5a6eb076d1/ios/Party%20Line/API/Client.swift#L353-L359

Which last but not least calls this Daily-js API function, which afaik is correct:

https://github.com/daily-demos/party-line/blob/42fed09a9523ae5949da2d24d66f6e5a6eb076d1/react/public/ios-bridge.html#L273-L275

I'll have to do some more digging. 🤔

aidan-waite commented 3 years ago

Any luck with this? Is there any other info I could provide to help debug?

regexident commented 3 years ago

Any luck with this? Is there any other info I could provide to help debug?

Hi Aidan,

I somehow cannot reproduce the issue reliably, but when it happens it seems to correlate to this error message spilled to the Xcode console:

Error acquiring assertion: <Error Domain=RBSAssertionErrorDomain Code=3 "Required client entitlement is missing"
UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"MediaPlayback"
sourceEnvironment:"(null)">, NSLocalizedFailureReason=Required client entitlement is missing}>

[ProcessSuspension] 0x102cd29a0 - ProcessAssertion: Failed to acquire RBS MediaPlayback assertion 'WebKit Media
Playback' for process with PID 1791, error: Error Domain=RBSAssertionErrorDomain Code=3 "Required client entitlement
is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"MediaPlayback"
sourceEnvironment:"(null)">, NSLocalizedFailureReason=Required client entitlement is missing}

Could you do another run yourself (clearing the log before step 10) and post the relevant part of the log so we can check if we're talking about the same error?

aidan-waite commented 3 years ago

Hello,

Here are the logs I got from Party Line:

  1. Clear logs and tap unmute on iOS
2021-05-11 23:19:11.961200+0900 Party Line[72487:6618800] Running Javascript: setMicrophoneEnabled(true);
2021-05-11 23:19:11.969793+0900 Party Line[72487:6618800] userContentController(_:didReceive:)
2021-05-11 23:19:11.969857+0900 Party Line[72487:6618800] message.name: console
2021-05-11 23:19:11.969883+0900 Party Line[72487:6618800] handleConsoleMessage(body:)
2021-05-11 23:19:11.970064+0900 Party Line[72487:6618800] WebView Console: "[TRACK STARTED]"
2021-05-11 23:19:11.971317+0900 Party Line[72487:6618800] userContentController(_:didReceive:)
2021-05-11 23:19:11.971369+0900 Party Line[72487:6618800] message.name: console
2021-05-11 23:19:11.971390+0900 Party Line[72487:6618800] handleConsoleMessage(body:)
2021-05-11 23:19:11.972330+0900 Party Line[72487:6618800] Error decoding console message: The data couldn’t be read because it isn’t in the correct format.
2021-05-11 23:19:11.972541+0900 Party Line[72487:6618800] userContentController(_:didReceive:)
2021-05-11 23:19:11.972565+0900 Party Line[72487:6618800] message.name: console
2021-05-11 23:19:11.972583+0900 Party Line[72487:6618800] handleConsoleMessage(body:)
2021-05-11 23:19:11.972711+0900 Party Line[72487:6618800] WebView Console: "[PARTICIPANT UPDATED]"
2021-05-11 23:19:11.972846+0900 Party Line[72487:6618800] userContentController(_:didReceive:)
2021-05-11 23:19:11.972865+0900 Party Line[72487:6618800] message.name: console
2021-05-11 23:19:11.972881+0900 Party Line[72487:6618800] handleConsoleMessage(body:)
2021-05-11 23:19:11.973743+0900 Party Line[72487:6618800] Error decoding console message: The data couldn’t be read because it isn’t in the correct format.
2021-05-11 23:19:11.974042+0900 Party Line[72487:6618800] userContentController(_:didReceive:)
2021-05-11 23:19:11.974065+0900 Party Line[72487:6618800] message.name: events
2021-05-11 23:19:11.974105+0900 Party Line[72487:6618800] handleEventMessage(body:)
2021-05-11 23:19:11.974201+0900 Party Line[72487:6618800] JSON: {"action":"participant-updated","participant":{"session_id":"dad4d353-060e-4a91-ed6d-759165af13d0","user_name":"aidan phone test_LST","user_id":"dad4d353-060e-4a91-ed6d-759165af13d0","joined_at":"2021-05-11T14:18:14.682Z","local":true,"owner":false,"will_eject_at":"2021-05-11T14:27:08.082Z","audio":true,"video":false,"screen":false,"tracks":{"audio":{"state":"playable","track":{"_managedByDaily":true},"persistentTrack":{"_managedByDaily":true}},"video":{"state":"interrupted"},"screenVideo":{"state":"off","off":{"byUser":true},"persistentTrack":null},"screenAudio":{"state":"off","off":{"byUser":true},"persistentTrack":null}},"cam_info":{},"screen_info":{},"record":false,"audioTrack":{"_managedByDaily":true}},"callFrameId":"16207426899440.46039637118236687"}
Event: participant-updated => Optional(ParticipantUpdatedEvent(participant: <Participant isLocal: true, userID: dad4d353-060e-4a91-ed6d-759165af13d0, isHandRaised: false, isOwner: false, sessionID: dad4d353-060e-4a91-ed6d-759165af13d0, username: aidan phone test, role: listener, isMicrophoneEnabled: true>))
2021-05-11 23:19:11.974888+0900 Party Line[72487:6618800] handleParticipantUpdatedEvent(_:)
  1. Tap mute
2021-05-11 23:19:25.096248+0900 Party Line[72487:6618800] Running Javascript: setMicrophoneEnabled(false);
2021-05-11 23:19:25.118864+0900 Party Line[72487:6618800] userContentController(_:didReceive:)
2021-05-11 23:19:25.118952+0900 Party Line[72487:6618800] message.name: console
2021-05-11 23:19:25.118988+0900 Party Line[72487:6618800] handleConsoleMessage(body:)
2021-05-11 23:19:25.119224+0900 Party Line[72487:6618800] WebView Console: "[TRACK STOPPED]"
2021-05-11 23:19:25.133670+0900 Party Line[72487:6618800] userContentController(_:didReceive:)
2021-05-11 23:19:25.133758+0900 Party Line[72487:6618800] message.name: console
2021-05-11 23:19:25.133788+0900 Party Line[72487:6618800] handleConsoleMessage(body:)
2021-05-11 23:19:25.134813+0900 Party Line[72487:6618800] Error decoding console message: The data couldn’t be read because it isn’t in the correct format.
2021-05-11 23:19:25.140083+0900 Party Line[72487:6618800] userContentController(_:didReceive:)
2021-05-11 23:19:25.140154+0900 Party Line[72487:6618800] message.name: console
2021-05-11 23:19:25.140180+0900 Party Line[72487:6618800] handleConsoleMessage(body:)
2021-05-11 23:19:25.140347+0900 Party Line[72487:6618800] WebView Console: "could not emit"
2021-05-11 23:19:25.195923+0900 Party Line[72487:6618800] userContentController(_:didReceive:)
2021-05-11 23:19:25.195990+0900 Party Line[72487:6618800] message.name: console
2021-05-11 23:19:25.196016+0900 Party Line[72487:6618800] handleConsoleMessage(body:)
2021-05-11 23:19:25.196178+0900 Party Line[72487:6618800] WebView Console: "[PARTICIPANT UPDATED]"
2021-05-11 23:19:25.240842+0900 Party Line[72487:6618800] userContentController(_:didReceive:)
2021-05-11 23:19:25.240906+0900 Party Line[72487:6618800] message.name: console
2021-05-11 23:19:25.240931+0900 Party Line[72487:6618800] handleConsoleMessage(body:)
2021-05-11 23:19:25.242075+0900 Party Line[72487:6618800] Error decoding console message: The data couldn’t be read because it isn’t in the correct format.
2021-05-11 23:19:25.242772+0900 Party Line[72487:6618800] userContentController(_:didReceive:)
2021-05-11 23:19:25.242807+0900 Party Line[72487:6618800] message.name: events
2021-05-11 23:19:25.242826+0900 Party Line[72487:6618800] handleEventMessage(body:)
2021-05-11 23:19:25.242851+0900 Party Line[72487:6618800] JSON: {"action":"participant-updated","participant":{"session_id":"dad4d353-060e-4a91-ed6d-759165af13d0","user_name":"aidan phone test_LST","user_id":"dad4d353-060e-4a91-ed6d-759165af13d0","joined_at":"2021-05-11T14:18:14.682Z","local":true,"owner":false,"will_eject_at":"2021-05-11T14:27:08.082Z","audio":false,"video":false,"screen":false,"tracks":{"audio":{"state":"off","off":{"byUser":true},"persistentTrack":{"_managedByDaily":true}},"video":{"state":"interrupted"},"screenVideo":{"state":"off","off":{"byUser":true},"persistentTrack":null},"screenAudio":{"state":"off","off":{"byUser":true},"persistentTrack":null}},"cam_info":{},"screen_info":{},"record":false},"callFrameId":"16207426899440.46039637118236687"}
Event: participant-updated => Optional(ParticipantUpdatedEvent(participant: <Participant sessionID: dad4d353-060e-4a91-ed6d-759165af13d0, isMicrophoneEnabled: false, isOwner: false, isLocal: true, isHandRaised: false, userID: dad4d353-060e-4a91-ed6d-759165af13d0, username: aidan phone test, role: listener>))
2021-05-11 23:19:25.246987+0900 Party Line[72487:6618800] handleParticipantUpdatedEvent(_:)

And here's the logs from the app I'm working on that shows the same TRACK STOPPED event and "could not emit" error. It has the same issue where I can no longer hear audio playback from the call when I mute myself. Note it's setup to be unmuted by default.

  1. Clear log and tap the mute button for the first time on iOS
"[TRACK STOPPED]"
Error decoding console message: The data couldn’t be read because it isn’t in the correct format.
"could not emit"
"[PARTICIPANT UPDATED]"
Error decoding console message: The data couldn’t be read because it isn’t in the correct format.
UpdatedEvent <Participant userID: 67dec485-fb4b-4b9e-e908-d8c22c7ff5f8, username: aidan+may30@mindsea.com, isLocal: true, sessionID: 67dec485-fb4b-4b9e-e908-d8c22c7ff5f8, isOwner: false, isMicrophoneEnabled: false>
  1. Clear again and tap unmute
"[TRACK STARTED]"
Error decoding console message: The data couldn’t be read because it isn’t in the correct format.
"[PARTICIPANT UPDATED]"
Error decoding console message: The data couldn’t be read because it isn’t in the correct format.
UpdatedEvent <Participant userID: 67dec485-fb4b-4b9e-e908-d8c22c7ff5f8, username: aidan+may30@mindsea.com, isLocal: true, sessionID: 67dec485-fb4b-4b9e-e908-d8c22c7ff5f8, isOwner: false, isMicrophoneEnabled: true>
aidan-waite commented 3 years ago

I re-downloaded the Party Line demo with the latest changes, deleted and re-installed the app, and tested it again and the issue remains.

regexident commented 3 years ago

Hm, so the Error decoding console message errors shouldn't really have an effect here, since they are just a bug in the forwarding of Javascript's console.log(…) to Swift.

I've opened https://github.com/daily-demos/party-line/pull/13 to fix the decoding errors, as well as https://github.com/daily-demos/party-line/pull/14 to fix an issue with showing/hiding mic/hand buttons.

Could you pull these fixes and see if you're still getting any errors logged? I'd be particularly interested in what gets logged on the phone when after 10) you mute/unmute the browser client.

aidan-waite commented 3 years ago

I tested fc6facfd130189c6e78d8045c913d4187d79ab8a and the I'm still experiencing the issue. Here are the logs:

  1. iOS tap unmute
2021-05-12 01:57:03.473786+0900 Party Line[73134:6663900] Running Javascript: setMicrophoneEnabled(true);
2021-05-12 01:57:03.491616+0900 Party Line[73134:6663900] userContentController(_:didReceive:)
2021-05-12 01:57:03.491723+0900 Party Line[73134:6663900] message.name: console
2021-05-12 01:57:03.491768+0900 Party Line[73134:6663900] handleConsoleMessage(body:)
2021-05-12 01:57:03.492117+0900 Party Line[73134:6663900] WebView Console: "[TRACK STARTED]"
2021-05-12 01:57:03.494614+0900 Party Line[73134:6663900] userContentController(_:didReceive:)
2021-05-12 01:57:03.494690+0900 Party Line[73134:6663900] message.name: console
2021-05-12 01:57:03.494721+0900 Party Line[73134:6663900] handleConsoleMessage(body:)
2021-05-12 01:57:03.497039+0900 Party Line[73134:6663900] WebView Console: ["track": ["_managedByDaily": true], "action": "track-started", "participant": ["screen_info": [:], "audioTrack": ["_managedByDaily": true], "user_id": "9868d780-51ce-4c65-86eb-01b3e6ad463e", "local": true, "owner": false, "cam_info": [:], "user_name": "aa bb_SPK", "audio": true, "joined_at": "2021-05-11T16:56:02.138Z", "will_eject_at": "2021-05-11T16:59:59.837Z", "session_id": "9868d780-51ce-4c65-86eb-01b3e6ad463e", "tracks": ["screenVideo": ["persistentTrack": null, "state": "off", "off": ["byUser": true]], "audio": ["persistentTrack": ["_managedByDaily": true], "track": ["_managedByDaily": true], "state": "playable"], "video": ["state": "interrupted"], "screenAudio": ["off": ["byUser": true], "state": "off", "persistentTrack": null]], "record": false, "screen": false, "video": false]]
2021-05-12 01:57:03.497465+0900 Party Line[73134:6663900] userContentController(_:didReceive:)
2021-05-12 01:57:03.497499+0900 Party Line[73134:6663900] message.name: console
2021-05-12 01:57:03.497527+0900 Party Line[73134:6663900] handleConsoleMessage(body:)
2021-05-12 01:57:03.497680+0900 Party Line[73134:6663900] WebView Console: "[PARTICIPANT UPDATED]"
2021-05-12 01:57:03.497847+0900 Party Line[73134:6663900] userContentController(_:didReceive:)
2021-05-12 01:57:03.497873+0900 Party Line[73134:6663900] message.name: console
2021-05-12 01:57:03.497894+0900 Party Line[73134:6663900] handleConsoleMessage(body:)
2021-05-12 01:57:03.500467+0900 Party Line[73134:6663900] WebView Console: ["callFrameId": "16207521505710.6118936941531493", "action": "participant-updated", "participant": ["screen_info": [:], "audioTrack": ["_managedByDaily": true], "user_id": "9868d780-51ce-4c65-86eb-01b3e6ad463e", "local": true, "owner": false, "cam_info": [:], "user_name": "aa bb_SPK", "audio": true, "session_id": "9868d780-51ce-4c65-86eb-01b3e6ad463e", "joined_at": "2021-05-11T16:56:02.138Z", "record": false, "tracks": ["screenAudio": ["persistentTrack": null, "state": "off", "off": ["byUser": true]], "audio": ["persistentTrack": ["_managedByDaily": true], "track": ["_managedByDaily": true], "state": "playable"], "screenVideo": ["off": ["byUser": true], "persistentTrack": null, "state": "off"], "video": ["state": "interrupted"]], "will_eject_at": "2021-05-11T16:59:59.837Z", "screen": false, "video": false]]
2021-05-12 01:57:03.500903+0900 Party Line[73134:6663900] userContentController(_:didReceive:)
2021-05-12 01:57:03.500934+0900 Party Line[73134:6663900] message.name: events
2021-05-12 01:57:03.501192+0900 Party Line[73134:6663900] handleEventMessage(body:)
2021-05-12 01:57:03.501233+0900 Party Line[73134:6663900] JSON: {"action":"participant-updated","participant":{"session_id":"9868d780-51ce-4c65-86eb-01b3e6ad463e","user_name":"aa bb_SPK","user_id":"9868d780-51ce-4c65-86eb-01b3e6ad463e","joined_at":"2021-05-11T16:56:02.138Z","local":true,"owner":false,"will_eject_at":"2021-05-11T16:59:59.837Z","audio":true,"video":false,"screen":false,"tracks":{"audio":{"state":"playable","track":{"_managedByDaily":true},"persistentTrack":{"_managedByDaily":true}},"video":{"state":"interrupted"},"screenVideo":{"state":"off","off":{"byUser":true},"persistentTrack":null},"screenAudio":{"state":"off","off":{"byUser":true},"persistentTrack":null}},"cam_info":{},"screen_info":{},"record":false,"audioTrack":{"_managedByDaily":true}},"callFrameId":"16207521505710.6118936941531493"}
Event: participant-updated => Optional(ParticipantUpdatedEvent(participant: <Participant isOwner: false, sessionID: 9868d780-51ce-4c65-86eb-01b3e6ad463e, role: speaker, isHandRaised: false, isMicrophoneEnabled: true, userID: 9868d780-51ce-4c65-86eb-01b3e6ad463e, isLocal: true, username: aa bb>))
2021-05-12 01:57:03.502835+0900 Party Line[73134:6663900] handleParticipantUpdatedEvent(_:)
  1. iOS tap mute
2021-05-12 01:57:26.555388+0900 Party Line[73134:6663900] Running Javascript: setMicrophoneEnabled(false);
2021-05-12 01:57:26.562751+0900 Party Line[73134:6663900] userContentController(_:didReceive:)
2021-05-12 01:57:26.562840+0900 Party Line[73134:6663900] message.name: console
2021-05-12 01:57:26.562875+0900 Party Line[73134:6663900] handleConsoleMessage(body:)
2021-05-12 01:57:26.563265+0900 Party Line[73134:6663900] WebView Console: "[TRACK STOPPED]"
2021-05-12 01:57:26.563545+0900 Party Line[73134:6663900] userContentController(_:didReceive:)
2021-05-12 01:57:26.563584+0900 Party Line[73134:6663900] message.name: console
2021-05-12 01:57:26.563608+0900 Party Line[73134:6663900] handleConsoleMessage(body:)
2021-05-12 01:57:26.565680+0900 Party Line[73134:6663900] WebView Console: ["participant": ["screen_info": [:], "user_id": "9868d780-51ce-4c65-86eb-01b3e6ad463e", "local": true, "owner": false, "cam_info": [:], "user_name": "aa bb_SPK", "audio": false, "session_id": "9868d780-51ce-4c65-86eb-01b3e6ad463e", "joined_at": "2021-05-11T16:56:02.138Z", "will_eject_at": "2021-05-11T16:59:59.837Z", "tracks": ["screenVideo": ["persistentTrack": null, "state": "off", "off": ["byUser": true]], "audio": ["state": "off", "off": ["byUser": true], "persistentTrack": ["_managedByDaily": true]], "video": ["state": "interrupted"], "screenAudio": ["state": "off", "off": ["byUser": true], "persistentTrack": null]], "record": false, "screen": false, "video": false], "action": "track-stopped", "track": ["_managedByDaily": true]]
2021-05-12 01:57:26.569655+0900 Party Line[73134:6663900] userContentController(_:didReceive:)
2021-05-12 01:57:26.569724+0900 Party Line[73134:6663900] message.name: console
2021-05-12 01:57:26.569756+0900 Party Line[73134:6663900] handleConsoleMessage(body:)
2021-05-12 01:57:26.569975+0900 Party Line[73134:6663900] WebView Console: "could not emit"
2021-05-12 01:57:26.572523+0900 Party Line[73134:6663900] userContentController(_:didReceive:)
2021-05-12 01:57:26.572599+0900 Party Line[73134:6663900] message.name: console
2021-05-12 01:57:26.572633+0900 Party Line[73134:6663900] handleConsoleMessage(body:)
2021-05-12 01:57:26.572831+0900 Party Line[73134:6663900] WebView Console: "[PARTICIPANT UPDATED]"
2021-05-12 01:57:26.575376+0900 Party Line[73134:6663900] userContentController(_:didReceive:)
2021-05-12 01:57:26.575442+0900 Party Line[73134:6663900] message.name: console
2021-05-12 01:57:26.575473+0900 Party Line[73134:6663900] handleConsoleMessage(body:)
2021-05-12 01:57:26.577989+0900 Party Line[73134:6663900] WebView Console: ["callFrameId": "16207521505710.6118936941531493", "action": "participant-updated", "participant": ["screen_info": [:], "user_id": "9868d780-51ce-4c65-86eb-01b3e6ad463e", "local": true, "owner": false, "cam_info": [:], "user_name": "aa bb_SPK", "audio": false, "session_id": "9868d780-51ce-4c65-86eb-01b3e6ad463e", "joined_at": "2021-05-11T16:56:02.138Z", "will_eject_at": "2021-05-11T16:59:59.837Z", "tracks": ["audio": ["off": ["byUser": true], "persistentTrack": ["_managedByDaily": true], "state": "off"], "screenAudio": ["state": "off", "persistentTrack": null, "off": ["byUser": true]], "screenVideo": ["persistentTrack": null, "state": "off", "off": ["byUser": true]], "video": ["state": "interrupted"]], "record": false, "screen": false, "video": false]]
2021-05-12 01:57:26.581328+0900 Party Line[73134:6663900] userContentController(_:didReceive:)
2021-05-12 01:57:26.581392+0900 Party Line[73134:6663900] message.name: events
2021-05-12 01:57:26.581426+0900 Party Line[73134:6663900] handleEventMessage(body:)
2021-05-12 01:57:26.581462+0900 Party Line[73134:6663900] JSON: {"action":"participant-updated","participant":{"session_id":"9868d780-51ce-4c65-86eb-01b3e6ad463e","user_name":"aa bb_SPK","user_id":"9868d780-51ce-4c65-86eb-01b3e6ad463e","joined_at":"2021-05-11T16:56:02.138Z","local":true,"owner":false,"will_eject_at":"2021-05-11T16:59:59.837Z","audio":false,"video":false,"screen":false,"tracks":{"audio":{"state":"off","off":{"byUser":true},"persistentTrack":{"_managedByDaily":true}},"video":{"state":"interrupted"},"screenVideo":{"state":"off","off":{"byUser":true},"persistentTrack":null},"screenAudio":{"state":"off","off":{"byUser":true},"persistentTrack":null}},"cam_info":{},"screen_info":{},"record":false},"callFrameId":"16207521505710.6118936941531493"}
Event: participant-updated => Optional(ParticipantUpdatedEvent(participant: <Participant isOwner: false, isMicrophoneEnabled: false, sessionID: 9868d780-51ce-4c65-86eb-01b3e6ad463e, role: speaker, isHandRaised: false, userID: 9868d780-51ce-4c65-86eb-01b3e6ad463e, username: aa bb, isLocal: true>))
2021-05-12 01:57:26.582801+0900 Party Line[73134:6663900] handleParticipantUpdatedEvent(_:)
regexident commented 3 years ago

Interesting. I'm not seeing any relevant errors in there. I was expecting one of these to surface. 🤔

Would it be possible for you to try this:

I'd be particularly interested in what gets logged on the phone when after 10) you mute/unmute the browser (i.e. Mac) client.

?

aidan-waite commented 3 years ago

I cleared logs on phone, then unmuted Mac for the first time, then muted Mac. I could hear the playback as expected, it seemed from my phone's point of view that everything worked correctly. The error you mentioned is present though: Required client entitlement is missing

2021-05-18 23:40:47.505363+0900 Party Line[16296:1626022] userContentController(_:didReceive:)
2021-05-18 23:40:47.505579+0900 Party Line[16296:1626022] message.name: console
2021-05-18 23:40:47.505651+0900 Party Line[16296:1626022] handleConsoleMessage(body:)
2021-05-18 23:40:47.506135+0900 Party Line[16296:1626022] WebView Console: "[TRACK STARTED]"
2021-05-18 23:40:47.506636+0900 Party Line[16296:1626022] userContentController(_:didReceive:)
2021-05-18 23:40:47.506691+0900 Party Line[16296:1626022] message.name: console
2021-05-18 23:40:47.506745+0900 Party Line[16296:1626022] handleConsoleMessage(body:)
2021-05-18 23:40:47.511082+0900 Party Line[16296:1626022] WebView Console: ["track": [:], "action": "track-started", "participant": ["user_id": "6d6b445b-b97b-487c-efcf-ed79be38f0ba", "tracks": ["screenVideo": ["subscribed": true, "state": "off", "persistentTrack": [:], "off": ["byUser": true]], "screenAudio": ["subscribed": true, "state": "off", "persistentTrack": [:], "off": ["byUser": true]], "video": ["track": [:], "persistentTrack": [:], "state": "playable", "subscribed": true], "audio": ["subscribed": true, "persistentTrack": [:], "state": "playable", "track": [:]]], "screen": false, "audioTrack": [:], "owner": true, "record": false, "local": false, "screen_info": [:], "session_id": "6d6b445b-b97b-487c-efcf-ed79be38f0ba", "videoTrack": [:], "video": true, "audio": true, "cam_info": [:], "joined_at": "2021-05-18T14:37:55.384Z", "will_eject_at": "2021-05-18T14:47:49.291Z", "user_name": "mac test_MOD"]]
2021-05-18 23:40:47.818216+0900 Party Line[16296:1626022] userContentController(_:didReceive:)
2021-05-18 23:40:47.818312+0900 Party Line[16296:1626022] message.name: console
2021-05-18 23:40:47.818339+0900 Party Line[16296:1626022] handleConsoleMessage(body:)
2021-05-18 23:40:47.818521+0900 Party Line[16296:1626022] WebView Console: "[PARTICIPANT UPDATED]"
2021-05-18 23:40:47.822165+0900 Party Line[16296:1626022] userContentController(_:didReceive:)
2021-05-18 23:40:47.822215+0900 Party Line[16296:1626022] message.name: console
2021-05-18 23:40:47.822236+0900 Party Line[16296:1626022] handleConsoleMessage(body:)
2021-05-18 23:40:47.824456+0900 Party Line[16296:1626022] WebView Console: ["action": "participant-updated", "callFrameId": "16213487828360.6045772688482132", "participant": ["tracks": ["screenAudio": ["subscribed": true, "state": "off", "off": ["byUser": true], "persistentTrack": [:]], "video": ["subscribed": true, "state": "playable", "track": [:], "persistentTrack": [:]], "audio": ["state": "playable", "persistentTrack": [:], "subscribed": true, "track": [:]], "screenVideo": ["subscribed": true, "persistentTrack": [:], "off": ["byUser": true], "state": "off"]], "will_eject_at": "2021-05-18T14:47:49.291Z", "session_id": "6d6b445b-b97b-487c-efcf-ed79be38f0ba", "video": true, "screen_info": [:], "record": false, "cam_info": [:], "audioTrack": [:], "screen": false, "local": false, "joined_at": "2021-05-18T14:37:55.384Z", "audio": true, "user_id": "6d6b445b-b97b-487c-efcf-ed79be38f0ba", "owner": true, "user_name": "mac test_MOD", "videoTrack": [:]]]
2021-05-18 23:40:47.828548+0900 Party Line[16296:1626022] userContentController(_:didReceive:)
2021-05-18 23:40:47.828602+0900 Party Line[16296:1626022] message.name: events
2021-05-18 23:40:47.828699+0900 Party Line[16296:1626022] handleEventMessage(body:)
2021-05-18 23:40:47.828727+0900 Party Line[16296:1626022] JSON: {"action":"participant-updated","participant":{"session_id":"6d6b445b-b97b-487c-efcf-ed79be38f0ba","user_name":"mac test_MOD","user_id":"6d6b445b-b97b-487c-efcf-ed79be38f0ba","joined_at":"2021-05-18T14:37:55.384Z","local":false,"owner":true,"will_eject_at":"2021-05-18T14:47:49.291Z","audio":true,"video":true,"screen":false,"tracks":{"audio":{"subscribed":true,"state":"playable","track":{},"persistentTrack":{}},"video":{"subscribed":true,"state":"playable","track":{},"persistentTrack":{}},"screenVideo":{"subscribed":true,"state":"off","off":{"byUser":true},"persistentTrack":{}},"screenAudio":{"subscribed":true,"state":"off","off":{"byUser":true},"persistentTrack":{}}},"cam_info":{},"screen_info":{},"record":false,"audioTrack":{},"videoTrack":{}},"callFrameId":"16213487828360.6045772688482132"}
Event: participant-updated => Optional(ParticipantUpdatedEvent(participant: <Participant username: mac test, userID: 6d6b445b-b97b-487c-efcf-ed79be38f0ba, role: moderator, sessionID: 6d6b445b-b97b-487c-efcf-ed79be38f0ba, isOwner: true, isMicrophoneEnabled: true, isLocal: false, isHandRaised: false>))
2021-05-18 23:40:47.829964+0900 Party Line[16296:1626022] handleParticipantUpdatedEvent(_:)
2021-05-18 23:40:47.940754+0900 Party Line[16296:1626022] [assertion] Error acquiring assertion: <Error Domain=RBSAssertionErrorDomain Code=3 "Required client entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"MediaPlayback" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Required client entitlement is missing}>
2021-05-18 23:40:47.940891+0900 Party Line[16296:1626022] [ProcessSuspension] 0x1064d3300 - ProcessAssertion: Failed to acquire RBS MediaPlayback assertion 'WebKit Media Playback' for process with PID 16298, error: Error Domain=RBSAssertionErrorDomain Code=3 "Required client entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"MediaPlayback" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Required client entitlement is missing}
2021-05-18 23:40:47.941736+0900 Party Line[16296:1626022] [assertion] Error acquiring assertion: <Error Domain=RBSAssertionErrorDomain Code=3 "Required client entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"MediaPlayback" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Required client entitlement is missing}>
2021-05-18 23:40:47.942112+0900 Party Line[16296:1626022] [ProcessSuspension] 0x1064d3330 - ProcessAssertion: Failed to acquire RBS MediaPlayback assertion 'WebKit Media Playback' for process with PID 16296, error: Error Domain=RBSAssertionErrorDomain Code=3 "Required client entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"MediaPlayback" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Required client entitlement is missing}
2021-05-18 23:41:17.203775+0900 Party Line[16296:1626022] userContentController(_:didReceive:)
2021-05-18 23:41:17.203915+0900 Party Line[16296:1626022] message.name: console
2021-05-18 23:41:17.203979+0900 Party Line[16296:1626022] handleConsoleMessage(body:)
2021-05-18 23:41:17.204364+0900 Party Line[16296:1626022] WebView Console: "[TRACK STOPPED]"
2021-05-18 23:41:17.204848+0900 Party Line[16296:1626022] userContentController(_:didReceive:)
2021-05-18 23:41:17.204905+0900 Party Line[16296:1626022] message.name: console
2021-05-18 23:41:17.204948+0900 Party Line[16296:1626022] handleConsoleMessage(body:)
2021-05-18 23:41:17.208995+0900 Party Line[16296:1626022] WebView Console: ["participant": ["user_name": "mac test_MOD", "session_id": "6d6b445b-b97b-487c-efcf-ed79be38f0ba", "screen_info": [:], "local": false, "user_id": "6d6b445b-b97b-487c-efcf-ed79be38f0ba", "screen": false, "cam_info": [:], "will_eject_at": "2021-05-18T14:47:49.291Z", "videoTrack": [:], "joined_at": "2021-05-18T14:37:55.384Z", "owner": true, "record": false, "tracks": ["audio": ["subscribed": true, "persistentTrack": [:], "state": "off", "off": ["byUser": true]], "screenVideo": ["state": "off", "persistentTrack": [:], "subscribed": true, "off": ["byUser": true]], "screenAudio": ["subscribed": true, "off": ["byUser": true], "state": "off", "persistentTrack": [:]], "video": ["persistentTrack": [:], "subscribed": true, "track": [:], "state": "playable"]], "video": true, "audio": false], "action": "track-stopped", "track": [:]]
2021-05-18 23:41:17.216864+0900 Party Line[16296:1626022] userContentController(_:didReceive:)
2021-05-18 23:41:17.216949+0900 Party Line[16296:1626022] message.name: console
2021-05-18 23:41:17.216987+0900 Party Line[16296:1626022] handleConsoleMessage(body:)
2021-05-18 23:41:17.217228+0900 Party Line[16296:1626022] WebView Console: "could not emit"
2021-05-18 23:41:17.220435+0900 Party Line[16296:1626022] userContentController(_:didReceive:)
2021-05-18 23:41:17.220508+0900 Party Line[16296:1626022] message.name: console
2021-05-18 23:41:17.220543+0900 Party Line[16296:1626022] handleConsoleMessage(body:)
2021-05-18 23:41:17.220749+0900 Party Line[16296:1626022] WebView Console: "[PARTICIPANT UPDATED]"
2021-05-18 23:41:17.223918+0900 Party Line[16296:1626022] userContentController(_:didReceive:)
2021-05-18 23:41:17.223994+0900 Party Line[16296:1626022] message.name: console
2021-05-18 23:41:17.224029+0900 Party Line[16296:1626022] handleConsoleMessage(body:)
2021-05-18 23:41:17.227930+0900 Party Line[16296:1626022] WebView Console: ["callFrameId": "16213487828360.6045772688482132", "action": "participant-updated", "participant": ["user_name": "mac test_MOD", "session_id": "6d6b445b-b97b-487c-efcf-ed79be38f0ba", "screen_info": [:], "local": false, "user_id": "6d6b445b-b97b-487c-efcf-ed79be38f0ba", "screen": false, "cam_info": [:], "will_eject_at": "2021-05-18T14:47:49.291Z", "videoTrack": [:], "joined_at": "2021-05-18T14:37:55.384Z", "owner": true, "record": false, "tracks": ["screenVideo": ["persistentTrack": [:], "subscribed": true, "off": ["byUser": true], "state": "off"], "audio": ["subscribed": true, "off": ["byUser": true], "state": "off", "persistentTrack": [:]], "screenAudio": ["subscribed": true, "off": ["byUser": true], "state": "off", "persistentTrack": [:]], "video": ["state": "playable", "track": [:], "subscribed": true, "persistentTrack": [:]]], "video": true, "audio": false]]
2021-05-18 23:41:17.230357+0900 Party Line[16296:1626022] userContentController(_:didReceive:)
2021-05-18 23:41:17.230790+0900 Party Line[16296:1626022] message.name: events
2021-05-18 23:41:17.230837+0900 Party Line[16296:1626022] handleEventMessage(body:)
2021-05-18 23:41:17.230877+0900 Party Line[16296:1626022] JSON: {"action":"participant-updated","participant":{"session_id":"6d6b445b-b97b-487c-efcf-ed79be38f0ba","user_name":"mac test_MOD","user_id":"6d6b445b-b97b-487c-efcf-ed79be38f0ba","joined_at":"2021-05-18T14:37:55.384Z","local":false,"owner":true,"will_eject_at":"2021-05-18T14:47:49.291Z","audio":false,"video":true,"screen":false,"tracks":{"audio":{"subscribed":true,"state":"off","off":{"byUser":true},"persistentTrack":{}},"video":{"subscribed":true,"state":"playable","track":{},"persistentTrack":{}},"screenVideo":{"subscribed":true,"state":"off","off":{"byUser":true},"persistentTrack":{}},"screenAudio":{"subscribed":true,"state":"off","off":{"byUser":true},"persistentTrack":{}}},"cam_info":{},"screen_info":{},"record":false,"videoTrack":{}},"callFrameId":"16213487828360.6045772688482132"}
Event: participant-updated => Optional(ParticipantUpdatedEvent(participant: <Participant userID: 6d6b445b-b97b-487c-efcf-ed79be38f0ba, sessionID: 6d6b445b-b97b-487c-efcf-ed79be38f0ba, role: moderator, isMicrophoneEnabled: false, isHandRaised: false, username: mac test, isLocal: false, isOwner: true>))
2021-05-18 23:41:17.232163+0900 Party Line[16296:1626022] handleParticipantUpdatedEvent(_:)
aidan-waite commented 3 years ago

I updated to iOS 14.5.1 but I'm still seeing this issue. You said you can't reproduce the issue reliably. So muting on iOS sometimes works for you? I've never seen it work even once... could you share the OS version and iPhone model you've got it sometimes working on?

aidan-waite commented 3 years ago

I tried using https://partyline.daily.co/ on iOS in a WKWebView in an empty app and I don't get the mute/unmute issue.

aidan-waite commented 3 years ago

When I mute/unmute on iOS I am seeing the mute status update correctly from the Mac participant's POV. It seems that the issue is something to do with the iOS-specific setup. I tried matching the WKWebView settings of my app where I'm experiencing the mute bug to match the settings of the empty app test of https://partyline.daily.co/ but that doesn't seem to make a difference. So something to do with the iOS-specific setup that's not WKWebView configuration.

aidan-waite commented 3 years ago

I do seem to get this error on iOS when this issue occurs:

Error acquiring assertion
RBSAssertionErrorDomain Code=3 "Required client entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"MediaPlayback" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Required client entitlement is missing

So I think your earlier comment was on to something

aidan-waite commented 3 years ago

I think I fixed it!!

https://github.com/MindSea/Maui-Daily-Bridge/commit/af2c06c39a9f446db97e75f3f4c8c02ced79367a

I updated the bridge file to start/stop audio tracks based on user_id rather than sessionId. And mute/unmute works correctly đź‘Ť

I believe what was happening was I would mute on iOS, that would fire a track stopped event, and the bridge would nuke the only audio element, removing any audio from iOS.

regexident commented 3 years ago

This is great to hear, @aidan-waite!

I unfortunately don't have read-access to the repo you linked, though. Would you mind posting the relevant change/fix here as a code snippet?

aidan-waite commented 3 years ago

Sure:

Also you can see the whole html bridge file at: https://chat.99walks.fit

regexident commented 3 years ago

Oh, looks like I had a typo in there: using sessionId instead of session_id. (Your use of user_id is fine, too, of course.)

Fix: https://github.com/daily-demos/party-line/pull/16