dbuezas / icsee-ptz

56 stars 10 forks source link

CPU 100% alarm_worker #8

Closed dapuzz closed 1 year ago

dapuzz commented 1 year ago

Hi D, sorry to bother you, in your opinion, is it possible that alarm server is blocking in some way the home assistant instance? Sometimes, during the day, CPU spikes at 100% and I need to restart the container. No evidence inside the logs.

image The ps-spy top shows the situation above. Is it correct to have the alarm worker such higher? If you think this is by design please feel free to delete this issue.

Thanks again Dario

dbuezas commented 1 year ago

Hi! Very interesting, I don't understand why this could happen. The alarm_worker function is quite simple:

    async def alarm_worker(self):
        while self.socket_writer:
            await self.busy.acquire()
            try:
                (
                    head,
                    version,
                    session,
                    sequence_number,
                    msgid,
                    len_data,
                ) = struct.unpack("BB2xII2xHI", await self.socket_recv(20))
                await asyncio.sleep(0.1)  # Just for receive whole packet
                reply = await self.socket_recv(len_data)
                self.packet_count += 1
                reply = json.loads(reply[:-2])
                if msgid == self.QCODES["AlarmInfo"] and self.session == session:
                    if self.alarm_func is not None:
                        self.alarm_func(reply[reply["Name"]], sequence_number)
            except:
                pass
            finally:
                self.busy.release()
dbuezas commented 1 year ago

Oh, wait... if the socket is dead, this whole thing becomes a never ending loop

dbuezas commented 1 year ago

Ok, I checked, this should actually work fine. When the socket is closed, self.socket_writer is set to None, which should stop that loop

dbuezas commented 1 year ago

Is this still occurring for you in the last version?

dbuezas commented 1 year ago

I think this just happened to me, and then home assistant restarted itself

dbuezas commented 1 year ago

Ok, I can confirm this bug too

image

Commands to see this:

docker exec -it homeassistant bash
apk add py-spy --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/testing/ --allow-untrusted
py-spy top -- python3 -m homeassistant --config /config

I'll take a deeper look, this completely crashes home assistant!

dbuezas commented 1 year ago

v4.0.2 should fix this. Thanks for reporting this! I didn't know about py-spy, very useful!