Not possible to mute / unmute audio on iOS

[groyee]

Checklist:

• [v] I updated to the latest version available
• [v] I cleared the cache of my browser

Release with the issue:

• HA Core: 2024.3.1
• Operating System: 12.1
• Frontentd: 20240307.0

Last working release (if known):

Browser and Operating System: iOS 17.4

Description of problem:

Not sure if this is a bug or some kind of limitation but it is not possible to mute / unmute audio on iOS companion App in MSE mode. I see there is a mention in the documentation that the fullscreen option doesn't work in iOS but I don't see anything like that related to volume (mute and unmute). What even more strange that it does work in WebRTC mode but doesn't work in MSE mode.

From some reason in my case WebRTC doesn't work well, every few seconds the video stops and refreshes as if it can't buffer enough while MSE works fine. Also, I belive over 3G WebRTC not even available at all so MSE it is.

The thing is that the default media controls in MSE mode are not visible / enabled at all in MSE mode on iOS but it does work in Desktop. So at the end of the day, I can't see anything with audio because there is no button to unmute.

Also, if I select the custom frigate mute button in the menu it also doesn't show this button.

My Frigate card:

type: custom:frigate-card
cameras:
  - camera_entity: camera.entrance
    go2rtc:
      modes:
        - mse
    live_provider: go2rtc
view:
  default: live
  camera_select: live
  update_force: true
menu:
  buttons:
    fullscreen:
      priority: 60
      alignment: matching
      icon: mdi:fullscreen
    media_player:
      enabled: false
    expand:
      enabled: true
  style: hidden
live:
  preload: true
  controls:
    builtin: true
    timeline: {}
  microphone:
    always_connected: false
media_gallery:
  controls:
    thumbnails:
      show_details: true
performance:
  features:
    animated_progress_indicator: false
  profile: high

MSE Video on Desktop (Works as expected, volume button at the bottom right when clicking on the video):

WebRTC on iOS Mobile (Works as expected, volume button at the top right corner when clicking on the video):

MSE on iOS Mobile (Doesn't work, no mute / unmute button when clicking on the video):

Frigate card diagnostic information

Javascript errors shown in the web inspector (if applicable):

Additional information:

[groyee]

Just to add a bit more info. When I try to view using WebRTC Card from AlexxIT (Not using Frigate card) then MSE works as expected on iOS. I can see the default video controls including volume. So it can't be iOS limitation.

[groyee]

I will keep updating here :-) just in case more people have similar issues.

Even though with AlexxIT custom card I can see audio controls in MSE mode, they don't really do anything. I can't get audio to work.

Then I saw this: https://github.com/AlexxIT/go2rtc#codecs-madness

So... Looks like the problem is not with frigate card but with annoying Apple. MSE + IPhone = no audio.

I guess the next thing to do is to understand which ports to open so that WebRTC will work on 3G with HA

[NickM-27]

You can have audio in mse, you just need opus or pcm

[groyee]

Hm... I believe I already have audio configured for opus. Here is my config:

go2rtc:
  streams:
    entrance:
      - rtsp://user:password@192.168.1.84:554/stream1 # <- stream which supports video & aac audio.
      - "ffmpeg:entrance#audio=opus" # <- copy of the stream which transcodes audio to opus for webrtc

So, on iPhone, you can get audio in MSE mode in the frigate card?

[NickM-27]

yes, I have audio in mse on iphone

[groyee]

I see. That's already very helpful. Now I know there is a hope :-)

Any chance you can share your frigate.yml and the frigate card yaml? So I can compare the difference?