Unreliable WebRTC playback in Safari iOS

[bartvp1]

Describe the bug While streaming with RTMP, playback over WebRTC in Chromium based browsers is all good, but in Safari for iOS the video playback freezes a lot.

Encoder

• FFmpeg
• Quality: 1080p
• Codec: h264
• FPS: 30

Server

• OS: Ubuntu
• OvenMediaEngine Version: v0.16.5
• One Origin server for RTMP ingress
• Two Edge servers for WebRTC egress

Player

• Device: iPhone 11, 13, 15 (including but not exclusively)
• OS: iOS 17.4
• Browser: Safari
• OvenPlayer: v0.10.35

Player config:

OvenPlayer.create(playerId, {
      mute: true,
      controls: false,
      autoStart: true,
      sources: [
        {
          type: 'webrtc',
          file: streamUrl,
        },
      ],
    })

edge.xml

<?xml version="1.0" encoding="UTF-8"?>
<Server version="8">
    <Name>OvenMediaEngine</Name>
    <Type>edge</Type>
    <IP>*</IP>
    <StunServer>stun.l.google.com:19302</StunServer>

    <Bind>
        <Publishers>
            <WebRTC>
                <Signalling>
                    <Port>3333</Port>
                </Signalling>
                <IceCandidates>
                    <IceCandidate>*:10000-10005/udp</IceCandidate>
                    <TcpRelay>*:3478</TcpRelay>
                </IceCandidates>
            </WebRTC>
        </Publishers>
    </Bind>

    <VirtualHosts>
        <VirtualHost include="VHost*.xml" />
        <VirtualHost>
            <Name>default</Name>
            <Host>
                <Names>
                    <Name>${env:OME_HOSTNAME:*}</Name>
                    <Name>*.ome-edge</Name>
                    <Name>ome-edge</Name>
                    <Name>localhost</Name>
                </Names>
            </Host>

            <Origins>
                <Origin>
                    <Location>/app/</Location>
                    <Pass>
                        <Scheme>ovt</Scheme>
                        <Urls>
                            <Url>ome-origin:9000/app/</Url>
                        </Urls>
                    </Pass>
                </Origin>
            </Origins>

            <Applications>
                <Application>
                    <Name>app</Name>
                    <Type>live</Type>
                    <OutputProfiles>
                        <OutputProfile>
                            <Name>default</Name>
                            <OutputStreamName>${OriginStreamName}</OutputStreamName>

                            <Encodes>
                                <Video>
                                    <Bypass>true</Bypass>
                                </Video>
                                <Audio>
                                    <Bypass>true</Bypass>
                                </Audio>

                                <Video>
                                    <Codec>h264</Codec>
                                    <Height>1080</Height>
                                    <Bitrate>8000000</Bitrate>
                                    <Framerate>30</Framerate>
                                    <Preset>faster</Preset>
                                </Video>
                                <Video>
                                    <Codec>vp8</Codec>
                                    <Height>1080</Height>
                                    <Bitrate>8000000</Bitrate>
                                    <Framerate>30</Framerate>
                                    <Preset>faster</Preset>
                                </Video>
                                <Video>
                                    <Codec>h264</Codec>
                                    <Height>720</Height>
                                    <Bitrate>5000000</Bitrate>
                                    <Framerate>30</Framerate>
                                    <Preset>faster</Preset>
                                </Video>
                                <Video>
                                    <Codec>vp8</Codec>
                                    <Height>720</Height>
                                    <Bitrate>5000000</Bitrate>
                                    <Framerate>30</Framerate>
                                    <Preset>faster</Preset>
                                </Video>
                                <Video>
                                    <Codec>h264</Codec>
                                    <Height>360</Height>
                                    <Bitrate>2500000</Bitrate>
                                    <Framerate>30</Framerate>
                                    <Preset>faster</Preset>
                                </Video>
                                <Video>
                                    <Codec>vp8</Codec>
                                    <Height>360</Height>
                                    <Bitrate>2500000</Bitrate>
                                    <Framerate>30</Framerate>
                                    <Preset>faster</Preset>
                                </Video>
                                <Video>
                                    <Codec>h264</Codec>
                                    <Height>144</Height>
                                    <Bitrate>400000</Bitrate>
                                    <Framerate>30</Framerate>
                                    <Preset>faster</Preset>
                                </Video>
                                <Video>
                                    <Codec>vp8</Codec>
                                    <Height>144</Height>
                                    <Bitrate>400000</Bitrate>
                                    <Framerate>30</Framerate>
                                    <Preset>faster</Preset>
                                </Video>

                                <Audio>
                                    <Codec>opus</Codec>
                                    <Bitrate>128000</Bitrate>
                                    <Samplerate>48000</Samplerate>
                                    <Channel>2</Channel>
                                </Audio>
                            </Encodes>
                        </OutputProfile>
                    </OutputProfiles>

                    <Providers>
                        <OVT />
                    </Providers>

                    <Publishers>
                        <WebRTC>
                            <Timeout>30000</Timeout>
                        </WebRTC>
                    </Publishers>
                </Application>
            </Applications>
        </VirtualHost>
    </VirtualHosts>
</Server>

origin.xml

<?xml version="1.0" encoding="UTF-8" ?>
<Server version="8">
    <Name>OvenMediaEngine</Name>
    <Type>origin</Type>
    <IP>*</IP>
    <StunServer>stun.l.google.com:19302</StunServer>

    <Bind>
        <Providers>
            <RTMP>
                <Port>1935</Port>
            </RTMP>
        </Providers>

        <Publishers>
            <OVT>
                <Port>9000</Port>
            </OVT>
        </Publishers>
    </Bind>

    <VirtualHosts>
        <VirtualHost include="VHost*.xml" />
        <VirtualHost>
            <Name>default</Name>
            <Host>
                <Names>
                    <Name>${env:OME_HOSTNAME:*}</Name>
                    <Name>*.ome-origin</Name>
                    <Name>ome-origin</Name>
                    <Name>localhost</Name>
                </Names>
            </Host>

            <Applications>
                <Application>
                    <Name>app</Name>
                    <Type>live</Type>
                    <OutputProfiles>
                        <OutputProfile>
                            <Name>default</Name>
                            <OutputStreamName>${OriginStreamName}</OutputStreamName>
                            <Encodes>
                                <Video>
                                    <Bypass>true</Bypass>
                                </Video>
                                <Audio>
                                    <Bypass>true</Bypass>
                                </Audio>
                                <Audio>
                                    <Codec>opus</Codec>
                                    <Bitrate>128000</Bitrate>
                                    <Samplerate>48000</Samplerate>
                                    <Channel>2</Channel>
                                </Audio>
                            </Encodes>
                        </OutputProfile>
                    </OutputProfiles>

                    <Providers>
                        <RTMP>
                            <BlockDuplicateStreamName>true</BlockDuplicateStreamName>
                        </RTMP>
                    </Providers>

                    <Publishers>
                        <OVT />
                    </Publishers>
                </Application>
            </Applications>
        </VirtualHost>
    </VirtualHosts>
</Server>

[naanlizard]

Please share your player settings, server settings, server logs, encoder settings, and any other logs or settings you can get, to help us debug

[bartvp1]

I just added server & player configs & encoder settings to the original post

[getroot]

In Edge, Encodesare unconditionally bypassed. (OVT is a special protocol for edge distribution.) To apply “Encodes”, you must do so on the Origin server. And you are encoding and outputting 8 video tracks, which is using a lot of resources and is likely affecting performance. However, if you do not set up ABR using "Playlist", your Encodeswill not be used.

Anyway, for the above reasons, your Safari is playing what ffmpeg sends as is, and there may be a problem with codec compatibility. When sending video to ffmpeg, try encoding with H264 / Baseline profile / Zerolatency Tune / Ultrafase Preset.

[bartvp1]

The issue has been resolved by adding the encodings to the origin instead of the Edge. Thanks @getroot!