tsightler / ring-mqtt

Ring devices to MQTT Bridge
MIT License
586 stars 105 forks source link

Live Video Streaming #238

Closed tsightler closed 3 years ago

tsightler commented 3 years ago

So the day has come where pigs are flying and the cows are on their way home. I've been asked about inclusion of live streaming in this addon for, well, since pretty much the day I started supporting cameras and, while I've always said never, I guess never might have been a little too long because I'm now considering inclusion of live video support in ring-mqtt.

Now, of course, it won't be via MQTT, rather it will use an RTSP streaming server (the exact technology is still under evaluation, likely it will be the node based rtsp-streaming-server package, or the external rtsp-simple-server) and it will require manual creation of camera entities in Home Assistant (although you can also use it without HA, for example, via VLC or any other media player able to play streams) because I can't figure out any method for an addon to automatically create non-MQTT based entities without some type of helper HA based integration (if there is a way, someone please let me know, but there's always the possibility of a helper integration in the future if this stuff works OK).

There will of course be limits, as Ring limits sessions to ~10 minutes, so it's not going to be a full time live streaming session, but I still think it will be a major step up from the existing solutions and at least everything will be integrated in a single place. I'm also hoping to implement a creative way to playback the prior X number of recorded videos, with X still to be determined.

I'm currently looking for testers that are interested in giving the camera live streaming support a try and reporting any issues or bugs. This will involve simply being willing to run the dev version, making the required manual changes to your HA configs, and a willingness/ability to collect and provide logs for troubleshooting any issues. I hope to push the first dev version with live streaming support in the next week or so.

stephywephy88 commented 3 years ago

The live stream worked for me during dev testing, but the stream_source changed when the add-on was updated and now the streams don’t work. Is the stream_source supposed to look like this? In dev I had a “172.”… rtsp url

I didn’t know what data was sensitive, so I just scratched a lot out. Thank you again for your help and efforts :)

CF6ADD90-07C9-4715-B7D0-ABEEB20E95C4

Edit: I use supervised HAC add-on

tsightler commented 3 years ago

Technically the stream URLs did not change at all, you can still just use the IP address of the container if that works for you as this is what was displayed before. However, I didn't want to use that for the production code because I was concerned that the IP address could change at any time (I really don't know how consistent Docker is at handing out IP addresses, but it seems pretty random). On the other hand, Home Assistant generates the addon hostname using a documented and predictable algorithm and the local.hass.io domain is used only for resolution of internal HA instances so, if you are running the addon, the addon hostname should theoretically always work even if the IP address changes because the hostname should not change (same as using DHCP on a "normal" client).

That being said, there is something that looks weird in your setup perhaps indicating a bug in my code (I only have two test systems, but they both show correctly and resolve as expected). I would expect the container hostname for the ring-mqtt addon to always be 03cabcc9-ring-mqtt, because, as I mentioned above, it's computed from a stable algorithm by HA itself, but yours appear to start with a98 rather than 03c. Perhaps that's nothing, maybe you just used a slightly different URL for the Github repo or something, but it was surprising to me.

What would be interesting is, when you click on the addon and you are on the Info tab, it should show the hostname for the addon's container and you should be able to just take that and just use it for the streaming URL.

For example, if your hostname shown on the addon info tab is "03cabcc9-ring-mqtt", which is what I would have expected it to be, then the streaming URL is rtsp://03cabcc9-ring-mqtt:8554/_live (using the short name without the domain), or rtsp://03cabcc9-ring-mqtt.local.hass.io:8554/_live if using the FQDN used internally by HA. Note that right now the addon computes this using the local container's DNS (it basically does a reverse lookup of the container IP address) so I'm a little surprised that it could be wrong, but like I said above, I only have two test systems, there could easily be a bug hiding in there somewhere. I probably should just pull the hostname from the API when running as the addon but I was trying to use the same algorithm for both the addon and standard Docker installs.

If you have the Web & SSH Terminal addon installed so you can access to your HA instance console you should be able to ping the name in the hostname field to determine the IP address of the container and you can use that for the RTSP streaming URL as well, just be aware that it could always change in the future if you just use the IP. Of course, if your are able to ping the hostname to get the IP, then you should be able to just use that for the streaming URL.

oramirite commented 3 years ago

Docker container IPs change quite frequently, and the method you chose is a best practice for referencing other Docker containers in many setups. Hostname is the way to go, because that should resolve correctly for all Docker containers. Obviously having it be configurable goes a long way to letting us handle any edge cases too :)

I'm amazed at the great work you've done here; I finally got a chance to swap this out from the ring-hassio project I was already using to grab the live stream: https://github.com/jeroenterheerdt/ring-hassio. It's not too dissimilar to yours, but when I discovered the WebRTC card I realized this other addon wouldn't work since it's exposing a stream over http and not RTSP. Yours made for a nice drop-in replacement.

I was pleasantly surprised to find you'd already done this work and fully tested everything with WebRTC. Things are mostly working amazingly :) I'm using the WebRTC card in a browser_mod popup with an automation to automatically pop it up on a doorbell ring. This is an amazing combination and the delay is really, really low.

One issue I am starting to have now that everything's set up, is that the audio from the livestream will continue playing in the background after I close the window. It goes away after switching apps (android). This doesn't happen every time; I'm trying to see if I can find a way to reproduce.

I am using this as a stream in Frigate, instead of the generic camera. I haven't actually tried clip recording and I've disabled detection, so I'm not sure how often Frigate is actually streaming that camera. I ran this setup using the other Ring Livestream addon for at least 3 months 24/7... I won't do that with your plugin, at least not for now, but jsut something I wanted to throw in for research purposes.

Thank you for pursuing this; it's a great way to handle it and it's really taken this plugin up from a useful tool to a powerhouse of an addon. This is definitely the de-facto way to run a Ring camera on HA now.

tsightler commented 3 years ago

I am using this as a stream in Frigate, instead of the generic camera. I haven't actually tried clip recording and I've disabled detection, so I'm not sure how often Frigate is actually streaming that camera. I ran this setup using the other Ring Livestream addon for at least 3 months 24/7... I won't do that with your plugin, at least not for now, but jsut something I wanted to throw in for research purposes.

This is actually one of my biggest concerns. It's really easy to add automatic restart to the stream, it's actually built into rtsp-simple-server so it's just a parameter change, but I'm afraid a bunch of people will run it that way all the time and Ring won't be too happy about it and eventually block connections from non-authorized clients. To this point they've tolerated us hacking at their API.

That being said, I've noticed that even the Ring web app now allows the stream to auto-restart, and it just uses WebRTC itself. Also, they've been promising 24x7 streaming as a feature for a while, so maybe they don't care too much.

Regarding the audio issues, that's all frontend stuff so nothing to do on the backend for me, which is good because I need to take a break from this project for a few weeks, at least once I get the "play prior recordings" feature implemented, which I think I'm close to completing.

oramirite commented 3 years ago

I will disable this for now then. After looking at the "Stream Enable" switch (thanks for that) for the past 2 hours, it does reactivate no matter what I try. I'm assuming this is because Frigate accesses the camera 24/7. Disabling object detection doesn't seem to make a difference. I definitely want to respect your concerns here, especially since you just released this.

My suspicion would also be that they're very close to not caring, as it probably wouldn't be hard to throttle my account specifically. But I am happy to follow your comfort levels there. I don't really have a need for 24/7 streaming anyway, since the angle of my doorbell cam makes everything set off motion detection. I also have a second camera with a better overhead view of our street and front step, so the live view is really all I need form this camera. All I really want is a stick-up cam.

After doing a little more research the audio issue seems to be related to the WebRTC card actually. So I will look into that. I already discovered that deactivating the stream with your switch stops the audio though - so I might just automate that to stop when I close the popup and all should be good :)

You deserve a break and thanks again for this!!

oramirite commented 3 years ago

I actually just had a thought - if we could create some sort of low-res secondary stream that Frigate is compatible with (for example an RTSP output of the snapshots), then this could be used as the "detect" input of a Frigate camera and allow 24/7 "streaming" of this low-res view (Frigate allows you to configure the low-res stream of a camera for object detection).

This would allow the use of Frigate features like clip browsing without relying on the actual live stream being accessed 24/7. One could theoretically just automate the recordings manually with this setup, and maybe even base them off of the motion events of your MQTT sensors to rebuild the detection logic and trigger a Frigate clip recording. This, in theory, should activate the high-res livestream normally, since that's the feed Frigate switches over to when recording clips anyway.

Whew! A bit messy, and takes a little work on the user end, but in theory could work?

This would be a lot of work just to avoid 24/7 livestreaming... but since there's also the issue of motion events not working while the livestream is active maybe there would be a reason for it.

tsightler commented 3 years ago

I can pretty much assure you I won't be adding anything like this to ring-mqtt. The goal of this project is not to somehow fill every gap in Ring's feature set, it's to integrate Ring devices with existing Home Automation tools that people use. If you want a camera or even doorbell that works well with Frigate, then you should probably just buy something other than Ring's super closed ecosystem.

That being said, all the tools to do what you ask are already available, either by using ring-client-api directly, or by writing a script to use ring-mqtt and ffmpeg to grab snapshot images and send them to Frigate. I don't know if Frigate will find a 1 frame per 10 second low-res video useful, but you seem to know way more about it than I do.