search

potmat / homebridge-google-nest-sdm

A Homebridge plugin for Google Nest devices that uses the Google Smart Device Management API. Supports Cameras, Doorbells, Displays, and Thermostats. Includes support for HomeKit Secure Video.
ISC License
134 stars 15 forks source link

HKSV Causes Orphaned FFMpeg Processes, Leading to Ever Increasing Memory Consumption #150

Open TanoopSungha opened 5 months ago

TanoopSungha commented 5 months ago

After the last two updates I now find that it now gobbles up RAM on the Pi. When I restart the child bridge it then resolves the issue. I am using a Pi5.

potmat commented 5 months ago

That's very odd, I can't account for it as there were no changes to the plugin itself, merely the arguments it passes to ffmpeg. Can you see which process is using the RAM (Homebridge or ffmpeg)?

TanoopSungha commented 5 months ago

The memory only goes down when I reboot or restart the child bridge to google nest sdm - anything else doesn't change it or improve the situation.

potmat commented 4 months ago

Hi @TanoopSungha, I've not be able to reproduce this, my Pi4 still has 95% memory available after weeks of uptime. I'll leave it open if anyone can reproduce it and submit a PR.

hillaker commented 3 months ago

@potmat, I am seeing the same behavior as TanoopSungha on my Raspberry Pi 5-- memory slowly disappears over the course of the day until I have less than 0.5GB available (out of 8).

Please find a screenshot of top sorted by memory-- it looks like ffmpeg is sticking around after a particular recording is complete. Happy to share anything else that might help identify a fix-- I'm running Kali linux, rather than Raspian or similar, and it's the ARM64 version. Not sure if that's relevant for this issue:

image

Memory after letting the plugin run for ~9 hours (after opening a stream, I left it running without opening a stream earlier in the day and didn't notice an increase in memory usage:

image

Memory after restarting the plugin's bridge (but not home bridge), taken ~1 minute after the previous screenshot:

image

If there's something I can do to change my configuration to make it easier to diagnose the issue, please let me know and I'll give it a shot and share the results. Thanks!

Edited again to add that I only have this plugin as one that an outputs video-- my other plugins are switches.

GillesWi commented 3 months ago

Yes, same behaviour on Raspberry Pi 4 (4Gb). After 7 days the memory increases so much that Homebridge just crashes and I need to unplug the Rpi to get it working again. If I restart the plugin within those days there are no problems...

And a week ago I lost my whole homebridge instance, everything was corrupt, so be sure to backup in time ;)!

ShadowRegent commented 1 month ago

Can confirm that I'm also seeing orphaned ffmpeg processes consuming high amounts of memory. I had 16 processes with just 5 hours of uptime. Killing them (with SIGKILL, they were unresponsive to SIGTERM) released memory and left the following logs after killing:

[homebridge-google-nest-sdm] Encountered unexpected error on generator Error: FFMPEG socket closed during read for 8 bytes! at Socket.endHandler (/usr/local/lib/node_modules/homebridge-google-nest-sdm/src/HksvStreamer.ts:190:24) at Socket.emit (node:events:519:28) at TCP.<anonymous> (node:net:338:12)

This was preceded by many errors that looked like this:

Recording download stream 1 is still awaiting generator although stream was closed 10s ago! This is a programming mistake by the camera implementation which prevents freeing up resources.

potmat commented 1 month ago

@ShadowRegent: Ok that means that HKSV is what's causing the issue.

It's starting up streams to record motion (or whatever) events and the FFMpeg process are not getting properly cleaned up.

To everyone in the thread, turning off HKSV will likely solve the issue. I know that's not great, but we can continue to investigate HKSV and hopefully find out why so many processes are being orphaned.