FPS drop using go2rtc stream over vpn

[ofcedwards]

Checklist:

• [x] I updated to the latest version available
• [x] I cleared the cache of my browser

Release with the issue: 5.2.0

Last working release (if known): Unknown

Browser and Operating System: Mobile App (iOS 17.4) / Safari (macOS Sonoma 14.4)

Description of problem: The frigate live stream using go2rtc plays at approximately 1 fps while using the frigate card over a vpn (wireguard). I found this to be repeatable on both iOS and macOS using both the Home Assistant app and Safari. My initial assumption was that my home upload speed was lacking, as it fluctuates as low as 20 mbit/s, but viewing the live stream on the frigate web page plays the video with no stutter or obvious reduction in fps. I found the issue to be slightly better using Firefox on macOS, where it seems more like the live stream is having a buffering issue. On Firefox, the video will skip and then smoothly play a small section before skipping again.

I have zero playback issues while connected to my home wifi. Again, I suspect a bandwidth issue, but it's odd that I can consistently play the live stream with no issue on the frigate web page and consistently have an issue streaming through the frigate card.

Frigate card diagnostic information

{
  "ha_version": "2024.3.1",
  "card_version": "5.2.0",
  "browser": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15",
  "date": "2024-03-18T22:27:47.671Z",
  "frigate_version": {
    "6dde1ac169d5f4c7736f45a56fcf778d": "5.0.1/0.13.2-6476f8a"
  },
  "lang": "en",
  "timezone": "America/New_York",
  "git": {
    "build_version": "5.2.0-HEAD+g69249b6",
    "build_date": "Fri, 23 Jun 2023 15:26:26 GMT",
    "commit_date": "Thu, 22 Jun 2023 09:21:26 -0600"
  },
  "config": {
    "type": "custom:frigate-card",
    "cameras": [
      {
        "frigate": {
          "url": "http://10.x.x.199:5000"
        },
        "dependencies": {
          "cameras": [
            "front-door-low-res"
          ]
        },
        "go2rtc": {
          "modes": [
            "mse",
            "webrtc"
          ]
        },
        "camera_entity": "camera.front_door",
        "live_provider": "go2rtc"
      },
      {
        "camera_entity": "camera.front_door",
        "live_provider": "jsmpeg",
        "id": "front-door-low-res",
        "hide": true
      }
    ],
    "view": {
      "default": "live"
    },
    "media_viewer": {
      "controls": {
        "thumbnails": {
          "show_timeline_control": false
        },
        "next_previous": {
          "style": "none"
        }
      }
    },
    "menu": {
      "style": "outside",
      "buttons": {
        "image": {
          "enabled": false
        },
        "fullscreen": {
          "alignment": "opposing"
        },
        "media_player": {
          "enabled": false
        },
        "timeline": {
          "enabled": false
        },
        "camera_ui": {
          "alignment": "opposing"
        },
        "microphone": {
          "enabled": false
        },
        "mute": {
          "enabled": false
        },
        "expand": {
          "enabled": false
        },
        "play": {
          "enabled": false
        },
        "recordings": {
          "enabled": true
        },
        "clips": {
          "enabled": false
        },
        "cameras": {
          "enabled": false
        }
      }
    },
    "dimensions": {
      "aspect_ratio_mode": "dynamic"
    },
    "live": {
      "controls": {
        "next_previous": {
          "style": "none"
        },
        "thumbnails": {
          "show_timeline_control": false,
          "show_download_control": false,
          "mode": "none"
        },
        "title": {
          "mode": "none"
        }
      },
      "show_image_during_load": false
    },
    "media_gallery": {
      "controls": {
        "thumbnails": {
          "size": 175,
          "show_timeline_control": false,
          "show_download_control": false
        }
      }
    }
  }
}

Javascript errors shown in the web inspector (if applicable):

Additional information:

[dermotduffy]

When using a HA card (incl. this one) everything needs to be effectively proxied via the Home Assistant server. So one theory would be if there was some sort of network connectivity issue / slowness between you and the HA server (when you connect over VPN), but not between you and the Frigate server? Otherwise, I can't really explain why you'd see the difference you're describing.

[ofcedwards]

I wouldn't expect any speed difference with HA and frigate, unless there's some performance lost in proxying with HA. Both HA and frigate are running on the same server (and LAN), with HA in a VM and frigate in docker.

I did notice that switching the live provider to "home assistant video stream" produced a smoother video. I wouldn't think so, but would using go2rtc as the live provider use more data or other resources?

[ofcedwards]

This appears to be a RTC issue. I tried the stream through the WebRTC card (outside of the frigate card) and had the same issue. Forcing the stream to MSE resolves the problem (but I could not force it using go2rtc as the live provider). I've switched to WebRTC card as the live provider with "mode: mse" as an option.