Allow a non-Frigate go2rtc URL endpoint

[dm82m]

Checklist:

• [x] I updated to the latest version available
• [x] I cleared the cache of my browser

Release with the issue: Latest Frigate Card Last working release (if known):

Browser and Operating System: Google Chrome on MacOs

Description of problem:

Cant get 2 Way Audio working with the frigate-hass-card. It works directly through go2rtc. But with the frigate-hass-card the microphone button is not available even if I try to explicitly enable it.

Frigate card diagnostic information

{
  "ha_version": "2023.6.2",
  "card_version": "5.1.1",
  "browser": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36",
  "date": "2023-06-19T09:01:54.122Z",
  "frigate_version": {},
  "lang": "en",
  "timezone": "Europe/Berlin",
  "git": {
    "build_version": "5.1.1-HEAD+g2d977be",
    "build_date": "Sat, 20 May 2023 20:35:29 GMT",
    "commit_date": "Sat, 20 May 2023 13:32:32 -0700"
  },
  "config": {
    "type": "custom:frigate-card",
    "cameras": [
      {
        "camera_entity": "camera.dima_cam_hd_stream",
        "go2rtc": {
          "stream": "tapo",
          "modes": [
            "webrtc"
          ]
        }
      }
    ],
    "menu": {
      "buttons": {
        "microphone": {
          "enabled": true
        }
      },
      "style": "overlay"
    }
  }
}

Javascript errors shown in the web inspector (if applicable):

Additional information:

[dermotduffy]

It works directly through go2rtc.

That's the hard part in my experience! If the below doesn't work -- please tell me how you're verifying that and what the setup is.

The microphone button will show up when:

• It is enabled (as it is in your config).
• The browser has granted access to your microphone (i.e. you have not denied permission, perhaps verify your HA domain is allowed access in your browser?)
• The stream is provider by go2rtc live_provider. You have NOT done this, please set this in your config:

  cameras:
  - camera_entity: camera.dima_cam_hd_stream
  live_provider: go2rtc

• go2rtc is in webrtc mode (your config will correctly require this).

My own personal setup involves 2-way audio, Frigate card, and a Tapo C110 -- so this feels like a configuration that should work. One thing I have not personally tested is use of a custom stream (in your config), although I see no reason it would not just work out of the box.

[dm82m]

thanks for you quick response!

If I put that live_provider it results in "Could not get camera endpoint for this live provider (incomplete configuration?)."

My detailed setup is:

1. HomeAssistant-Tapo-Control that gives me the cam with entity "camera.dima_cam_hd_stream"
2. go2rtc Addon with go2rtc.yaml config:

   streams:
   tapo: tapo://admin:MD5_pass_hash@192.168.178.63

3. WebRTC Camera

My HA is running on https. If I switch to go2rtc and select "video+audio+microphone = two way audio from camera" and use webrtc.html with this link my two way audio works.

My frigate-hass-card is now configured like this:

type: custom:frigate-card
cameras:
  - camera_entity: camera.dima_cam_hd_stream
    live_provider: go2rtc
    go2rtc:
      modes:
        - webrtc
      stream: tapo
menu:
  buttons:
    microphone:
      enabled: true

I am wondering about this in your documentation "Only Frigate cameras are supported." - what does that exactly mean?

Any further ideas?

[dermotduffy]

Oh, I assumed camera.dima_cam_hd_stream was a Frigate camera entity. The card is not able to auto-detect the Frigate camera name from a non-Frigate camera entity (not surprising). You'd need to either change camera_entity: camera.dima_cam_hd_stream to the correct Frigate camera entity, or try something like this to manually configure the Frigate camera name which may also work:

cameras:
  - camera_entity: camera.dima_cam_hd_stream
    frigate:
      camera_name: NAME_GOES_HERE //  <-------------- Fix this
    live_provider: go2rtc
    go2rtc:
      modes:
        - webrtc
      stream: tapo

One other, more serious issue, is that it looks like you're using a custom go2rtc addon rather than the one that runs with Frigate. This will not work, since the card expects the traffic to be proxied via the Frigate integration, and it only knows how to proxy to the go2rtc that comes with Frigate.

It might be reasonable to request that the URL used for go2rtc be customizable, but as this is only useful if the person has setup https in front of everything (like you) and for some reason is not using the stock go2rtc, I didn't bother to do so yet.

[dermotduffy]

It might be reasonable to request that the URL used for go2rtc be customizable, but as this is only useful if the person has setup https in front of everything (like you) and for some reason is not using the stock go2rtc, I didn't bother to do so yet.

Trying to figure out what to do with this issue: Is this a feature you would like implemented?

[dm82m]

Absolutely, as it would remove the need to use frigate and be able to use your card with 2 way audio!

[mutasim]

+1.

This would be great! I started with go2rtc, managed to get the Reolink doorbell 2-way working. Then I tried with this card and realised I would have to get frigate setup to get it working. All setup now but would prefer to drop frigate if poss (I'm using Blue Iris for my NVR + Codeproject for AI detection).

[dermotduffy]

@mutasim OK! Thanks for the additional support for this. Just to be certain this would help though -- I'll note that if you access HA via https you must also have your separate go2rtc instance accessible via https for this to work, and you'd need to configure this yourself. Otherwise, your browser would likely reject the requests the card is making to a non-https endpoint.

[mutasim]

Got it - very useful context thanks @dermotduffy.

Presumably we’re talking https + public accessible or would https over LAN be enough?

I have an NGINX proxy setup for internal & external hostnames, so that would be fine for my usecase. If publicly accessible, I guess it would benefit from some level of auth on it to prevent unwanted access e.g. basic auth (?)

[dermotduffy]

LAN should be just fine, it shouldn't matter as long as the address of both the HA instance and the go2rtc URL you would give the card both use the same protocol (that is, both https or both http, but never mixed).

[felipecrs]

Perhaps you can leverage the WebRTC integration for that too? It seems that it creates a proxy between Home Assistant and the underlying go2rtc.

https://github.com/AlexxIT/WebRTC/blob/master/custom_components/webrtc/__init__.py#L189

I think this would be an acceptable compromise: if you want to use go2rtc outside of Frigate in the Frigate Card, you need the WebRTC integration installed and connected to this go2rtc.

[mutasim]

Happy to give that route a go - should that work as is?

[dermotduffy]

That shouldn't be required, though. As long as the go2rtc instance is accessible, and the protocol is the same, it should work (if I add card support for changing the URL). Note that this means, in order for the card to access go2rtc when you're remote, you'd need to expose go2rtc to the internet -- which would likely be a bad idea...

[dm82m]

WebRTC (webrtc-camera card) with underlying go2rtc also needs port 8555 (or own defined port) to be forwarded to HA. I don’t get the point what it means that you need to expose go2rtc to internet. If it is the described above I see no issue for me personally.

[felipecrs]

Port 8555 isn't the only one required, you would probably need to expose port 1854 as well, I think.

[felipecrs]

And that's problematic, I guess. I would very much prefer not to expose go2rtc to internet.

[dermotduffy]

WebRTC (webrtc-camera card) with underlying go2rtc also needs port 8555 (or own defined port) to be forwarded to HA. I don’t get the point what it means that you need to expose go2rtc to internet. If it is the described above I see no issue for me > personally.

In order for this to work, when you're remote (off your home network), the go2rtc instance needs to be accessible to the internet (i.e. the ports for go2rtc forward through your router). Otherwise, when you're away from home, go2rtc would not be accessible from the card. Forwarding go2rtc to the internet would be a very bad plan, as you could accidentally expose your video feeds to anyone / everyone unless you implemented some sort of security mechanism in front (which is for the advanced user).

In short: I think this can work fine on your local network, or remotely if you implement your own security (but not otherwise).

[daxcore]

+1

[felipecrs]

In short: I think this can work fine on your local network, or remotely if you implement your own security (but not otherwise).

What about https://github.com/dermotduffy/frigate-hass-card/issues/1206#issuecomment-1637329141? That way it should work remotely without having to implementing my own security.

[MSMikl]

+1 It's a very necessary feature, because allows us to use the two-way audio without Frigate NVR. I can't use Frigate with my Dorrbell, because streaming from it blocks dorrbell button. So, I'd like to open video with an audio only temporarily, and that's why I need Frigate card withou Frigate camera

[dermotduffy]

This is now implemented and will be in the next release. The new functionality includes:

• Specifying a custom go2rtc backend for Frigate cameras.
• Adding go2rtc support for non-Frigate cameras.

[pergolafabio]

Wow , that's great!! So it only consumes CPU when actually watching the camera?

[pergolafabio]

Does it also mean go2rtc is included like the webrtc card? Or do nee to run the go2rtc addon?

[felipecrs]

@pergolafabio yes for the first question as you no longer need Frigate. For the second, you will need the go2rtc add-on.

[pergolafabio]

Thnx, that's a nice addition!

[dm82m]

Great work @dermotduffy - is there any eta for the release or a early beta that can be tested?

[felipecrs]

You can always do this:

https://github.com/dermotduffy/frigate-hass-card/tree/dev?tab=readme-ov-file#advanced-users-installing-unreleased-versions

[pergolafabio]

Hey, installed the PR, did a clear temp files on browser, for card i added like this:

        - type: custom:frigate-card
          cameras:
            - camera_entity: camera.deurbel
              live_provider: go2rtc
              go2rtc:
                url: http://192.168.0.17:1984
                modes:
                  - webrtc

I also checked my go2rtc addon, the stream i working with 2 way audio:

streams:
  deurbel:
    - rtsp://syno:xxx@192.168.0.14:554/Sms=24.unicast
    - isapi://admin:xxx@192.168.0.70:80/

but i receive this:

Could not get camera endpoint for this live provider (incomplete configuration?). [Check troubleshooting](https://github.com/dermotduffy/frigate-hass-card#troubleshooting).
{
  "hide": false,
  "dependencies": {
    "all_cameras": false,
    "cameras": []
  },
  "triggers": {
    "motion": false,
    "occupancy": true,
    "entities": []
  },
  "engine": "auto",
  "frigate": {
    "client_id": "frigate"
  },
  "motioneye": {
    "images": {
      "directory_pattern": "%Y-%m-%d",
      "file_pattern": "%H-%M-%S"
    },
    "movies": {
      "directory_pattern": "%Y-%m-%d",
      "file_pattern": "%H-%M-%S"
    }
  },
  "live_provider": "go2rtc",
  "image": {
    "refresh_seconds": 1
  },
  "camera_entity": "camera.deurbel",
  "go2rtc": {
    "modes": [
      "webrtc"
    ]
  }
}

What am i missing?

[pergolafabio]

ok, the config is wrong, it shouyld be like this?

        - type: custom:frigate-card
          cameras:
            - go2rtc:
                url: http://192.168.0.17:1984
                stream: deurbel
                modes:
                  - webrtc

But i get:

Could not determine suitable engine for camera. [Check troubleshooting](https://github.com/dermotduffy/frigate-hass-card#troubleshooting).
{
  "hide": false,
  "dependencies": {
    "all_cameras": false,
    "cameras": []
  },
  "triggers": {
    "motion": false,
    "occupancy": true,
    "entities": []
  },
  "engine": "auto",
  "frigate": {
    "client_id": "frigate"
  },
  "motioneye": {
    "images": {
      "directory_pattern": "%Y-%m-%d",
      "file_pattern": "%H-%M-%S"
    },
    "movies": {
      "directory_pattern": "%Y-%m-%d",
      "file_pattern": "%H-%M-%S"
    }
  },
  "live_provider": "auto",
  "image": {
    "refresh_seconds": 1
  },
  "go2rtc": {
    "modes": [
      "webrtc"
    ],
    "stream": "deurbel"
  }
}

the stream name should be ok ?

[felipecrs]

Looks like you did not select go2rtc as the camera live stream engine.

[pergolafabio]

how do you mean? i added a go2rtc camera? i did it like here?

https://github.com/dermotduffy/frigate-hass-card/blob/5777b5b4c0bff0675d20b22cca2c607236f7ce73/README.md

[pergolafabio]

should it be like this then?

        - type: custom:frigate-card
          cameras:
            - camera_entity: camera.deurbel
              live_provider: go2rtc
              go2rtc:
                url: http://192.168.0.17:1984
                stream: deurbel
                modes:
                  - webrtc

[pergolafabio]

ok, got it running now after a restart and clear temp, but i dont see the "mic" button, also see this in my log

I'm accesssing my HA from my https external (cloudfmare), port 1984 is NOT open

Doesnt it work then? For webrtc card it wasl also not needed?

2023-10-09 15:48:08.201 ERROR (MainThread) [frontend.js.latest.202309110] Uncaught error from Chrome 117.0.0.0 on Windows 10
SecurityError: Failed to construct 'WebSocket': An insecure WebSocket connection may not be initiated from a page loaded over HTTPS.
g.onconnect (/hacsfiles/frigate-hass-card/live-go2rtc-6a0f301b.js:1:3348)
g.connectedCallback (/hacsfiles/frigate-hass-card/live-go2rtc-6a0f301b.js:1:1865)
apply (src/scoped-custom-element-registry.js:248:41)
W.O (/hacsfiles/frigate-hass-card/card-ceaf0e14.js:17:4802)
W.T (/hacsfiles/frigate-hass-card/card-ceaf0e14.js:17:4868)
W._$AI (/hacsfiles/frigate-hass-card/card-ceaf0e14.js:17:4664)
q.p (/hacsfiles/frigate-hass-card/card-ceaf0e14.js:17:3979)
W.$ (/hacsfiles/frigate-hass-card/card-ceaf0e14.js:17:5178)
W._$AI (/hacsfiles/frigate-hass-card/card-ceaf0e14.js:17:4634)
/hacsfiles/frigate-hass-card/card-ceaf0e14.js:23:657

[dermotduffy]

Great work @dermotduffy - is there any eta for the release or a early beta that can be tested?

On the v6 release, no ETA at the moment, quite a lot of work to do still.

Doesnt it work then? For webrtc card it wasl also not needed?

This is an immediate (<24h!) example of why I think people will be more disappointed and confused by this feature alone than delighted. This support is only useful if (1) this is for purely local trusted network access only, or (2) you have the networking expertise to setup a reverse proxy and authentication in front of your go2rtc instance -- which is dangerous if you get wrong.

Caveats in detail:

• Your browser will complain unless both your HA instance and the go2rtc url are both either http or https, you cannot mix.
• If you expect this to work remotely, whatever url you use needs to be remotely accessible. This is not a problem for the Frigate cameras or the WebRTC card (WebRTC "card" is actually an integration!), because both have integrations (code running in the HA server) that are doing the heavily lifting "internally to HA" of proxying your traffic via HA itself. Pure lovelace cards are not running in HA, they are running in your browser, so they cannot do this.

Solutions are:

• Have the networking expertise to set this up yourself.
• Perhaps there is, or could be, a generic "proxy" integration in HA.
• Perhaps the integration code within the "WebRTC card" can be reused, there are hints it might be extensible to create new links, but I haven't at all played with this.

[pergolafabio]

Hmm, ok thnx for feedback, i maybe need to wait a little bit longer to get this to work, how i wanted it too be :-)

I dont think its possible to run go2rtc in https mode, not sure about this

[dm82m]

trying to set it up for my local network but I am not successful. codebase is current dev branch.

given:

type: custom:frigate-card
cameras:
  - camera_entity: camera.dima_cam_hd_stream #provided by tapo integration
    live_provider: go2rtc
    go2rtc:
      url: http://192.168.178.4:1984
      stream: tapo
      modes:
        - webrtc
menu:
  buttons:
    microphone:
      enabled: true

[dermotduffy]

I dont think its possible to run go2rtc in https mode, not sure about this

You'd need to put a https reverse proxy in front of it (e.g. Apache, nginx). As above, not something the average user is necessarily going to be able to do. It wouldn't really make sense for go2rtc to support this natively, but it could theoretically.

trying to set it up for my local network but I am not successful. codebase is current dev branch.

Your config looks fine (you don't even need to provide a camera entity) -- I just tested it on my side and it works fine. Check your browser JS console (F12 in Chrome), you may have an error message in there. Possibly accessing your HA instance over https whilst you have given a http endpoint.

[dm82m]

no I am testing all with local http (not https). this is what the dev tools are showing:

[pergolafabio]

yes, same error here when accessing locally on http

[felipecrs]

From what I understood, you need to have go2rtc exposed as https. If you don't have a reverse proxy like NGINX Proxy Manager to help you achieve that, you can do it in go2rtc itself:

api:
  username: "admin"  # default "", Basic auth for WebUI
  password: "pass"   # default "", Basic auth for WebUI
  static_dir: "www"  # default "", folder for static files (custom web interface)
  tls_listen: ":443" # default "", enable HTTPS server
  tls_cert: |        # default "", PEM-encoded fullchain certificate for HTTPS
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----
  tls_key: |         # default "", PEM-encoded private key for HTTPS
    -----BEGIN PRIVATE KEY-----
    ...
    -----END PRIVATE KEY-----

The certificates can be generated using Let's Encrypt. Note that this is using a new version of go2rtc.

---

@dermotduffy have you taken a look on how difficult it would be for the card to leverage the WebRTC integration? Apparently it proxies go2rtc into HA's /api/webrtc/.

https://github.com/AlexxIT/WebRTC/blob/69cbcdafe09f6b3ae3dc0391073d499346324027/custom_components/webrtc/__init__.py#L178C12-L178C24

But I'm not sure if it proxies everything you need.

[pergolafabio]

aha, cool, didnt know that, i now have https running on port 1985, but i still have the same issue as @dm82m , when running also on http

[felipecrs]

@pergolafabio are you able to view go2rtc itself through that HTTPS address? I am not sure, but I believe a valid certificate is required.

[pergolafabio]

yeah, i tried that too, with the ap: i copy pasted the contents of my cert files, and also opened as a test 1985 on my router

i was able to surf by going to https://mydomainname:1985

[pergolafabio]

it also whos me a valid ssl , when i surf to that page if i enter the same url on frigate card, it still doesnt work

[pergolafabio]

in go2rtc addon i see this: 13:55:36.206 ERR github.com/AlexxIT/go2rtc/internal/api/ws/ws.go:101 > host=homeassistant.xxxx:1985 origin=https://homeassistant.xxxx:8123 error="websocket: request origin not allowed by Upgrader.CheckOrigin"

[felipecrs]

Add this to go2rtc:

api:
  origin: '*'

[pergolafabio]

cool, that works! :-)

ok, now i need to figure out a way it can be done without opening 1985 , maybe with a dns rewrite...

but would be great if it could be integrated natively

[pergolafabio]

would also be great if we could point the go2rtc to actually the cert files instead of copy/pasting the certificate :-)

[felipecrs]

would also be great if we could point the go2rtc to actually the cert files instead of copy/pasting the certificate :-)

I agree, I suggest you open the feature request in go2rtc.

[pergolafabio]

Done: https://github.com/AlexxIT/go2rtc/issues/680

[dermotduffy]

@dermotduffy have you taken a look on how difficult it would be for the card to leverage the WebRTC integration? Apparently it proxies go2rtc into HA's /api/webrtc/.

https://github.com/AlexxIT/WebRTC/blob/69cbcdafe09f6b3ae3dc0391073d499346324027/custom_components/webrtc/__init__.py#L178C12-L178C24

Yes actually! I spent 20 minutes playing on this yesterday, but couldn't get it to work out of the box. I do suspect there's a way though (instead I went and worked on your casting feature request!).

To me I think the right fix is really a "small generic proxy integration" designed for re-use. It could be implemented using a small common "forwarding" library, that this new integration, Frigate and WebRTC could all optionally share as a dependency. I'm sure there is a (small) set of other Lovelace cards that need this functionality too. I think it's important to separate the proxy from any usage of it to ensure it stays re-usable (i.e. this shouldn't be "Frigate proxy" or "WebRTC proxy" in my head).

I think the above is much better than asking every conceivable backend to support https, authentication, etc, etc -- which typically take a lot of investment to get "correct".