Open Horndrup opened 1 year ago
Hey. I couldn't find any information about the live camera protocol. I might need help getting this information
Yeah, the Mi Home app/API, that has been used until you and rovo89 managed to utilize the Roborock API, doesn't have camera support, so that's likely why no one has documented it.
I checked. The camera uses WebRTC. WebRTC must be available in Client Web. A connection is established only through a mutual handshake. Handshake is not public and can only be accessed by decompiling the app.
I am so thrilled that you are making this integration, i have been using miio app but never satisfied with that, now I'm back with roborock app that is great, and i will also be thrilled to get access to the live cam, and the speak through robot function, if you are able to do it, happy to helt test it, i am runnig a S6 MavV.
looking forward this feature
looking forward for this feature too
@AlexxIT did some great work in https://github.com/AlexxIT/WebRTC and https://github.com/AlexxIT/go2rtc, if anyone tries integration in HA check out those repos
Can someone provide how handshake process looks like? Maybe I can help. Ideally we need to make SDP exchange process.
Can someone provide how handshake process looks like? Maybe I can help. Ideally we need to make SDP exchange process.
Hi, I decompiled the app and found this part where it talks about WebRTC and SDP, is that what you are looking for?
You are welcome https://github.com/AlexxIT/go2rtc/releases/tag/v1.3.0
Wow great work @AlexxIT! I don't have a Roborock with a camera but I'll spend some time implementing this into the integration. I'll need a few testers
@humbertogontijo had let me know he doesn't have much time for hobby projects at the moment, and I've been working on turning this into a core integration, so I have a pretty good understanding of the code base
Thanks @AlexxIT, i will test this with my S7 MAXV Ultra and report back with my results.
I don't have a vacuum with a camera either. But I have thousands of users :) There is currently no support for reverse audio for the S6 MaxV model. I still plan to check what's wrong there.
Nice! I'll implement this in my roborock adapter for iobroker.
Looking at the go2rtc code, I don't see a straight forward way to add it into the roborock integration itself (that the core maintainers will accept when this part of the code gets added/ that I know of). + with me being unable to test anything I try, I'd rather leave this problem to someone else,.
I think for now it is definitely better for any user who wants access to the Roborock cameras to just download go2rtc, as it provides everything they need and can grab login from this integration. I'll create a PR to recommend as much in the Readme. And if anyone down the line has a better idea - they are more than welcome trying to implement it directly in the roborock integration - I believe the nest integration has done something similar in the past.
As I know, Hass support WebRTC cameras like Nest. This is about direct connection between user browser and camera. Hass in this case must exchange SDP and candidates. I've never experienced this part of Hass, so I can't help. But in this case the camera only works for viewing. You cannot record video from the camera or send a stream to Frigate.
This is amazing! I just need to first run
service: roborock.vacuum_remote_control_start
data: {}
target:
entity_id: vacuum.roborock_s7_maxv
So the vacuum leaves the dock, and it worked!
Great job @AlexxIT !
You can also add vacuum as media player and stream TTS and mp3 to it :) https://github.com/AlexxIT/WebRTC#stream-to-camera
Works only with S7 MaxV for now. You need to select opus codec.
Important - current webrtc integration has old go2rtc inside. You need to use it with external go2rtc.
Yeah - I saw Nest's implementation @AlexxIT
Camera Entities have a function: async def async_handle_web_rtc_offer(self, offer_sdp: str) -> str | None: which nest returns their "answer_sdp" to
However, looking at nest, basically all of the webRTC logic seems to be internal in their pypi package and not built directly into hass.
Since I know basically nothing about webrtc and I don't have a camera to test it on, I wasn't going to attempt it. I'm sure someone with webrtc know-how/ a roborock with a camera to test on could get it done, but it is outside my knowledge and what I have time for.
Perhaps @humbertogontijo can directly integrate it if he has time/ sees the need.
I tested the camera working with the latest go2rtc, great job! When I open the stream, the vacuuming stops and vacuum announces that the video call has been started. Is there a way to have the stream running while the vacuum is vacuuming like in the app?
Should I open feature request @AlexxIT ?
I think robot stops because audio. If you get rtsp://xxx:8554/robot1?video
or http://xxx:1984/webrtc.html?src=robot1&media=video
- go2rtc won't init audio stream.
I can add option to use only video for stream. So user can create two streams in go2rtc yaml. One with audio and one without.
The rtsp link worked flawlessly inside HA webrtc card, thank you very much!
What are the steps to see the frontcamera with go2rtc
What are the steps to see the frontcamera with go2rtc
streams:
camera.vacuum:
- roborock://..
If you want to add this to Lovelace install WebRTC Camera from HACS Add custom:webrtc-camera card add the rtsp url rtsp://x.x.x.x:8554/camera.vacuum?video <-- x.x.x.x Home Assistant IP
I think robot stops because audio. If you get
rtsp://xxx:8554/robot1?video
orhttp://xxx:1984/webrtc.html?src=robot1&media=video
- go2rtc won't init audio stream.I can add option to use only video for stream. So user can create two streams in go2rtc yaml. One with audio and one without.
Can you add the only audio option? I'm using a generic camera pointing to rtsp and shared to homekit. Now I can't use the vacuum because it always pauses the cleaning
I think robot stops because audio. If you get
rtsp://xxx:8554/robot1?video
orhttp://xxx:1984/webrtc.html?src=robot1&media=video
- go2rtc won't init audio stream.I can add option to use only video for stream. So user can create two streams in go2rtc yaml. One with audio and one without.
I tried both links on my S7 MaxV, both gives me connection refused, do I need to change something else? I can view the camera from the roborock app on my phone.
What is the URL format. I do not use hassio so I can not install the addon. I do have my token for roborock I'm just not sure on the url format
I think robot stops because audio. If you get
rtsp://xxx:8554/robot1?video
orhttp://xxx:1984/webrtc.html?src=robot1&media=video
- go2rtc won't init audio stream.I can add option to use only video for stream. So user can create two streams in go2rtc yaml. One with audio and one without.
Can you add the only audio option? I'm using a generic camera pointing to rtsp and shared to homekit. Now I can't use the vacuum because it always pauses the cleaning
Same for me, i tried with ?video but it still makes a call and stops the vacuum. Frustrating, the roborock addon also becomes unavailable for a while.
I think robot stops because audio. If you get
rtsp://xxx:8554/robot1?video
orhttp://xxx:1984/webrtc.html?src=robot1&media=video
- go2rtc won't init audio stream.I can add option to use only video for stream. So user can create two streams in go2rtc yaml. One with audio and one without.
Can you add the only audio option? I'm using a generic camera pointing to rtsp and shared to homekit. Now I can't use the vacuum because it always pauses the cleaning
Same for me, i tried with ?video but it still makes a call and stops the vacuum. Frustrating, the roborock addon also becomes unavailable for a while.
Actually it does work perfectly, only if i send the robot to a pin location does it still pause. So using ?video at the end of the url and adding media: video seems to solve the issue.
I think robot stops because audio. If you get
rtsp://xxx:8554/robot1?video
orhttp://xxx:1984/webrtc.html?src=robot1&media=video
- go2rtc won't init audio stream. I can add option to use only video for stream. So user can create two streams in go2rtc yaml. One with audio and one without.Can you add the only audio option? I'm using a generic camera pointing to rtsp and shared to homekit. Now I can't use the vacuum because it always pauses the cleaning
Same for me, i tried with ?video but it still makes a call and stops the vacuum. Frustrating, the roborock addon also becomes unavailable for a while.
Actually it does work perfectly, only if i send the robot to a pin location does it still pause. So using ?video at the end of the url and adding media: video seems to solve the issue.
HI ACS-LUX,
still struggling with the correct typo of the URL for my S7 MAXV .... my robot also stopps cleaning once i connect the remote viewing stream...
Could you please post how you URL looks like?
Mine is not working: rtsp://mqtt-eu-3.roborock.com:8883?HEREISMYTOKEN&pin=1234&video&media=video
BR
I think robot stops because audio. If you get
rtsp://xxx:8554/robot1?video
orhttp://xxx:1984/webrtc.html?src=robot1&media=video
- go2rtc won't init audio stream.I can add option to use only video for stream. So user can create two streams in go2rtc yaml. One with audio and one without.
I have the same problem as the others. When I load the stream, the vacuum stops the cleaning and states that it is paused for live stream.
You can also add vacuum as media player and stream TTS and mp3 to it :) https://github.com/AlexxIT/WebRTC#stream-to-camera
Works only with S7 MaxV for now. You need to select opus codec.
Important - current webrtc integration has old go2rtc inside. You need to use it with external go2rtc.
This is a nice feature, but I wish there was a way to keep the robot from pausing cleaning while audio is connected. I would very much like my robot to be able to play music while cleaning.
Same issue here as well. I can get the video stream, but it’ll stop the vacuums. Has anyone figured out the fix for this?
This question has been answered a long time ago. Only video should be requested from go2rtc, without sound.
This question has been answered a long time ago. Only video should be requested from go2rtc, without sound.
Even when it is requested as video only on go2rtc, it still doesn't work and pauses the clean cycle.
Maybe you doing something wrong. I don't have vacuum with camera, but other users said this is working method
I did several reboots and reloads of the integration and I can confirm it is now working as expected. It might have been related to the full RSPT URL that I was referencing. I also noticed that the WebRTC card would continue to load while the robot is docked. I'm using the blueprint automation to disable when docked, but I went ahead and used a conditional card for the cam card to only display when NOT docked. The other puzzle is the occasional issue where the extracted vacuum card doesn't display, but I think this is related to the API as it magically reappears as it did this morning for me. Things appear to work as expected. Thank you.
I did several reboots and reloads of the integration and I can confirm it is now working as expected. It might have been related to the full RSPT URL that I was referencing. I also noticed that the WebRTC card would continue to load while the robot is docked. I'm using the blueprint automation to disable when docked, but I went ahead and used a conditional card for the cam card to only display when NOT docked. The other puzzle is the occasional issue where the extracted vacuum card doesn't display, but I think this is related to the API as it magically reappears as it did this morning for me. Things appear to work as expected. Thank you.
@niveknow
I'm at my wits end. How did you get this to work? I'm using the following url format:
- roborock://mqtt-us.roborock.com:8883?u=xxxx&s=xxxx&k=xxxx&did=xxxx&key=xxxx&media=video&video
I tried rebooting the various integrations and restarting. It's still pausing the vacuum when I view the stream 😔
Thanks for all the great work. Everything is working fine.
I have a Roborock s6 maxV and would it be possible to see the stream from the front camera?