Support Tuya's WebRTC as source directly

[felipecrs]

Hi,

Some people in https://github.com/felipecrs/hass-expose-camera-stream-source/issues/5 has reported that Home Assistant did not have support for playing their Tuya cameras because of being WebRTC only.

I did some search about it, and I found the Tuya API SDK for it:

https://developer.tuya.com/en/docs/cloud/96c3154b0d?id=Kam7q5rz91dml

There is an endpoint for requesting the WebRTC stream with examples and so.

I believe it would be really nice if go2rtc supported Tuya cameras directly through their WebRTC relay. Maybe in the future even 2-way audio?

[felipecrs]

Oh, just to mention, to my knowledge this interface allows to play any Tuya camera, not only the WebRTC-only ones.

[AlexxIT]

If anyone repeats Tuya authorization in the golang, to the moment where I can get JSON from the link above - I'll add support. I don't want to waste time on the Tuya API.

[felipecrs]

Apparently Tuya itself distributes a golang client:

https://github.com/tuya/tuya-cloud-sdk-go

It seems to handle authorization, although I did not see a function to get the webrtc-configs JSON (at least not on a quick look in the README).

[felipecrs]

I also found this:

https://github.com/tuya/tuya-connector-go

[felipecrs]

Apparently the last one is more suitable, docs here:

https://developer.tuya.com/en/docs/iot/device-control-best-practice-go?id=Katymf1d9h4y6

[rodrigofragadf]

This would be greatly appreciated, there are many people having difficulties with Tuya cameras in the home assistant who would benefit from this solution!

https://github.com/home-assistant/core/issues/58614

[AlexxIT]

If someone adds Tuya/WebRTC to Home Assistant - go2rtc will be able to work with this camera https://github.com/AlexxIT/go2rtc/releases/tag/v1.6.0

[felipecrs]

That's a good idea!

[felipecrs]

Although, if implemented that way, I think 2-way audio will never be supported. :(

There are so many 2-way audio capable Tuya cameras and doorbells. There is this 12 USD doorbell for example:

https://aliexpress.com/item/1005005266340342.html

If we had 2-way audio support for such a thing someday in go2rtc, it would be amazing (Tuya's app sucks).

[AlexxIT]

All Tuya cameras, I've seen, are cheap garbage.

[felipecrs]

They definitely are cheap garbage. I no longer buy such cameras, but when I was not experienced and I didn't even know what Home Assistant is, I bought a (not cheap) Tuya doorbell and six (not cheap) Tuya cameras.

I think many people are in a similar situation, and any effort to make them be less garbage would be appreciated.

[felipecrs]

I found this BTW: https://github.com/tuya/webrtc-demo-go

But I wasn't able to make it work yet, struggling to get the auth code.

[felipecrs]

Ok, I'm kinda sad. Apparently using this API isn't free: https://www.reddit.com/r/homeassistant/comments/s6i27l/comment/ip1uzkk/?utm_source=share&utm_medium=web2x&context=3

But the same guy on this thread was able to workaround it, for free:

https://www.reddit.com/r/homeassistant/comments/s6i27l/i_managed_to_forward_the_tuyasmart_life_webrtc/?utm_source=share&utm_medium=web2x&context=3

[felipecrs]

I built a Bash script to get Tuya's WebRTC configs from Tuya API for some camera:

https://github.com/felipecrs/get-tuya-webrtc-configs

Not sure how to integrate with go2rtc yet though.

[felipecrs]

Well, it gets much harder. To be able to exchange SDP offer, a MQTT message needs to be sent to the Tuya IOT and then the result needs to be consumed from a MQTT subscription as well.

I'm trying to make this within the Bash script, but it's kinda crazy. I'm not sure if it's possible actually, as I need to send the SDP offer (which only go2rtc knows about).

I'm following this documentation: https://developer.tuya.com/en/docs/iot/webrtc?id=Kacsd4x2hl0se#title-9-Connect%20to%20WebRTC%20using%20signaling%20packets Which has an implementation example: https://github.com/tuya/webrtc-demo-go

[AlexxIT]

Roborock has same idea. SDP exchange ever MQTT with proprietary encryption.

[felipecrs]

I updated my gist with my attempt to make the MQTT negotiation. I was able to publish the MQTT topic successfully but I'm still unable to subscribe to get the response.

PS: For it to work, you need to add IOT Live Stream (or something like that) to your Tuya IOT subscription (but adding the subscription is free).

[felipecrs]

@AlexxIT do you think it is possible go2rtc to somehow pass the SDP offer to the echo script? Maybe something like:

# go2rtc.yaml

streams:
  my_camera:
    - echo:/config/scripts/get_tuya_webrtc_configs.sh <credentials> <camera id> {sdp_offer}

I believe the SDP offer is mandatory for Tuya's MQTT broker to reply.

[AlexxIT]

Yes. I can write special webrtc/bash source. Share your results. You may do it in PM

[felipecrs]

I have no results to share unfortunately, I'm able to publish the MQTT topic but not able to subscribe to get the MQTT response. My guess is that I'm not publishing a valid SDP and that's why I can't receive any results.

I'm sure it's possible because their demo application works, and it does basically the same thing.

If you are keen to implement such webrtc/bash source, please do it as it may be useful for other cases too. Then I can test.

[AlexxIT]

I need working code to connect to.

[felipecrs]

If anyone else wants to take the lead on this, go ahead... my last attempt is: https://github.com/felipecrs/get-tuya-webrtc-configs

Ideally it should be implemented directly in go2rtc as a webrtc/tuya source, the Bash script was just a rapid prototyping.

[felipecrs]

PS: This is confirmed to work: https://github.com/tuya/webrtc-demo-go

[berobloom]

I would be very much happy if someone could figure this out. My camera: LSC Indoor Camera 1080P does have an RTSP stream included by the vendor, but no audio... I tried https://github.com/tuya/webrtc-demo-go. Both the 2-way audio and video. works. If only someone could hack it so that only local communication is possible. :sunglasses:

[eerison]

Hey @felipecrs

I guess you get this working, or am I wrong?

[felipecrs]

Nope.

[eerison]

Nope.

and what about this https://github.com/felipecrs/get-tuya-webrtc-configs 🤔 ?

[felipecrs]

[eerison]

Do you have any exemple using tuya api? I can get turn and stun credentials, But I am not sure how to configure it on go2rtc :/

[pergolafabio]

If you request the rtsp in tuna API, do you get a response? Or do you only have stun/turn

[eerison]

If you request the rtsp in tuna API, do you get a response? Or do you only have stun/turn

I tired rtsps but it doesn't work :( I got the url, but no success, then I tried to configure the webrtc

Webrtc I get user, credentials, turn and stun urls

[felipecrs]

@AlexxIT this should be all required to make go2rtc work, no?

[pergolafabio]

Can go2rtc work with only the stun/turn info? I don't have the rtsp from the API, only turn/stun...

[pergolafabio]

when i do an api request on tuya website, i get this for example back, can this be used to fetch camera stream?

"p2p_config": {
      "auth": "xxx",
      "ices": [
        {
          "urls": "stun:aws3stun.tuyaeu.com:3478"
        },
        {
          "urls": "stun:3.126.77.24:3478"
        },
        {
          "credential": "xxx",
          "ttl": 3600,
          "urls": "turn:35.207.170.65:3478",
          "username": "xxx:xxx"
        }
      ],
      "moto_id": "signaling14901"

[AlexxIT]

This is only part of WebRTC connection. You need to do SDP exchange.

[eerison]

This is only part of WebRTC connection. You need to do SDP exchange.

do you have any example 👀 ?

[AlexxIT]

https://github.com/AlexxIT/go2rtc/blob/master/www/webrtc-sync.html

[eerison]

https://github.com/AlexxIT/go2rtc/blob/master/www/webrtc-sync.html

I couldn't make this work :'(

should I create this html file on my HA server? and how do I link this file with go2rtc.yaml file ?

[eerison]

PS: This is confirmed to work: https://github.com/tuya/webrtc-demo-go

I could make this work, But did you could make this work on go2rtc also? and it is working on localhost only :'(

[pergolafabio]

Hey, what are you trying to accomplish? I also have a tuya webcam (pet feeder) and for a very long time, the tuya api never provided me the rtsp link, also in HA, the screen was always black... Until suddenly a few weeks ago, something must have changed on tuya side, and the rtsp link was created... also now feed in HA with tuya integration

I use felipecrs his custom component to insert the stream into go2rtc

[eerison]

Hey, what are you trying to accomplish? I also have a tuya webcam (pet feeder) and for a very long time, the tuya api never provided me the rtsp link, also in HA, the screen was always black... Until suddenly a few weeks ago, something must have changed on tuya side, and the rtsp link was created... also now feed in HA with tuya integration

I use felipecrs his custom component to insert the stream into go2rtc

I am trying to integrate my doorbell with go2rtc.

I use felipecrs his custom component to insert the stream into go2rtc

do you have any step by step how to make this works?

[pergolafabio]

i use this one: https://github.com/felipecrs/hass-expose-camera-stream-source

But do you have the camera working in HA with tuya integration? thats mandatory

do you see an succes here? wirh rtsp ?

[eerison]

But do you have the camera working in HA with tuya integration? thats mandatory

But do you have the camera working in HA with tuya integration? thats mandatory

yep, but it is really slow :/

do you see an succes here? wirh rtsp ?

yep

i use this one:

I will try this one.

[pergolafabio]

the tuya integration is indeed slow, but that custom compnent makes it faster :-)

[eerison]

the tuya integration is indeed slow, but that custom compnent makes it faster :-)

Nice, it worked <3, thank you for your help.

But what config did you use to make this work 2-way audio 👀 ?

[felipecrs]

There is no way to make 2-way audio work with Tuya cameras right now. Maybe when direct WebRTC support gets added to go2rtc.

[eerison]

There is no way to make 2-way audio work with Tuya cameras right now. Maybe when direct WebRTC support gets added to go2rtc.

ahhhh ok.

just to point out, it works with 2-way audio 👀

https://github.com/tuya/webrtc-demo-go

[felipecrs]

That's great news! :)