home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
73.47k stars 30.69k forks source link

Unable to (find a way to) Exclude ZHA Events from Logbook #84590

Open teskanoo opened 1 year ago

teskanoo commented 1 year ago

The problem

Since I switched from Deconz to ZHA, my logbook is overrun with entries created by the ZHA Integration

I have found no way to prevent these entries from appearing in the logbook (which makes the logbook very difficult to read)

image

Having over 100 Zigbee devices this results in the Logbook consisting of about 70-75% ZHA Events with the remaining 25-30% being those events in which I have expressed an interest in being reported

I have attempted to exclude ZHA events in the logbook configuration by domain


logbook:
  exclude:
    domains:
    - zha

I have attempted to exclude ZHA events in the logbook via recorder configuration where I can exclude event_types

recorder:
  exclude:
    event_types:
    - zha_event
    - zha.zha_event

Recorder config, IMO, should be the solution to remove these logbook entries - I can only imagine that I am targeting the wrong event_types, however this is how they appear in my (MariaDB) Database

I have attempted to prevent the logging in logbook by reducing the logging level for the integration (in the vain hope that this might be a factor)

logger:
  default: info
  logs:
    homeassistant.components.zha: critical

I have verified that this issue occurs regardless of the Enable/Disable debug logging setting for the integration

image

I have attempted to identify in the codebase if and where any recorder and logbook exclusions are taken into consideration when the deciding to log into the logbook

What version of Home Assistant Core has the issue?

2022.12.8

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

ZHA, Logbook

Link to integration documentation on our website

https://www.home-assistant.io/integrations/zha/

Diagnostics information

No response

Example YAML snippet

Snippets from recorder and logbook configurations that I have tried to exclude these events from the logbook

recorder:
  exclude:
    event_types:
    - zha_event
    - zha.zha_event

logbook:
  exclude:
    domains:
    - zha


### Anything in the logs that might be useful for us?

_No response_

### Additional information

Forum post concerning this matter (which has been  present since August 2021 but has not, as far as I can see, been raised as an issue)

https://community.home-assistant.io/t/exclude-zha-events-from-logbook/452501/13
home-assistant[bot] commented 1 year ago

Hey there @dmulcahey, @adminiuga, @puddly, mind taking a look at this issue as it has been labeled with an integration (zha) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `zha` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Change the title of the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign zha` Removes the current integration label and assignees on the issue, add the integration domain after the command.

(message by CodeOwnersMention)


zha documentation zha source (message by IssueLinks)

jp8311 commented 1 year ago

+1 this issue, the lack of exclude filtering for the logbook in the ui, either temporary or 'save preferences' style, makes it very difficult to track a series of events and limits its usefulness. Now that I have a fairly large zigbee network I'm finding zha checkin event spam to create a ton of noise.

doekwerk commented 1 year ago

+1 dito

AndreFjeld commented 1 year ago

Same here, extremly annoying. image

doekwerk commented 1 year ago

I noticed that with @AndreFjeld and @Teskanoo have Zigbee motion sensors that appear in the Logbook. With me it is only the Hue switches that appear unwanted in the Logbook. My motion sensors and lights do not have this problem. They are in the domain light and binary_sensor. My Hue switches however are in the zha domain and are stateless devices. They cause an zha_event. These switches don't have an entity as such only for battery, update and binary_input. Since it has no entity we are not able to exclude it...

Hope someone knows how to change that.

Scherm­afbeelding 2023-01-10 om 21 01 18
snel6424 commented 1 year ago

+1 also having this issue. Would love to get these out of the Logbook

Yardco commented 1 year ago

Wondering why the ZHA integration is reporting these events in the log to begin with, not really seeing the relevance. I'd imagine that the zwavejs integration also checks in with zwave devices, but it doesn't display these events at all. I'd get it if I would have enabled debug logging or something but that's not the case.

doekwerk commented 1 year ago

recorder.yaml.txt I connected my Hue bridge V2 again and paired a few Hue switches to this bridge. Same resultaat. A Hue_event for each buttonpress and release in the logbook and db while hue_event type is excluded in the recorder config under event_types. I have Hue RWL021 and RWL022 switches. Maybe not a ZHA but recorder issue? Or the log code is the same for ZHA and Hue.

doekwerk commented 1 year ago

The logbook and recorder settings are the same: #72728 This is also explained in the logbook documentation. Problem: the logbook does only allow entities to be excluded and not events. Try: Logbook: exclude: event_events:

home-assistant[bot] commented 1 year ago

Hey there @home-assistant/core, mind taking a look at this issue as it has been labeled with an integration (logbook) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `logbook` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Change the title of the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign logbook` Removes the current integration label and assignees on the issue, add the integration domain after the command.

(message by CodeOwnersMention)


logbook documentation logbook source (message by IssueLinks)

snel6424 commented 1 year ago

Wondering why the ZHA integration is reporting these events in the log to begin with, not really seeing the relevance. I'd imagine that the zwavejs integration also checks in with zwave devices, but it doesn't display these events at all. I'd get it if I would have enabled debug logging or something but that's not the case.

My ZWave JS is reporting these events as well

image
doekwerk commented 1 year ago

Current logbook behavoir is that when I do a page refresh the ZHA and Hue events which should not be there are gone from the logbook. They are not written to the DB so the recorder ZHA and Hue events exclude works. So only in the live feed. This was discussed last year in another thread leading to #72728

Yardco commented 1 year ago

Wondering why the ZHA integration is reporting these events in the log to begin with, not really seeing the relevance. I'd imagine that the zwavejs integration also checks in with zwave devices, but it doesn't display these events at all. I'd get it if I would have enabled debug logging or something but that's not the case.

My ZWave JS is reporting these events as well image

With the difference being that you interacted with the device right? I’d deem these different (and more useful) then the redundant checkin events Zha is logging, which seem to only indicate that it tried to check if the device is still there.

snel6424 commented 1 year ago

That does seem to be the case, although I would still like them to not be there

teskanoo commented 1 year ago

Current logbook behavoir is that when I do a page refresh the ZHA and Hue events which should not be there are gone from the logbook. They are not written to the DB so the recorder ZHA and Hue events exclude works. So only in the live feed. This was discussed last year in another thread leading to #72728

Thanks doekwerk - you nailed it there

These events were definitely being added (interwoven) into the logbook AFTER the logbook page loads - it is something to do with the "previously introduced live logbook" mentioned on this page

https://rc.home-assistant.io/blog/2023/01/25/release-20232/#streaming-history

JonLaliberte commented 1 year ago

This one is annoying. I also wonder if the logbook changes could be causing this: https://github.com/home-assistant/core/issues/87128

douglasmauro commented 1 year ago

Why limit this to ZHA events? It would be great if we could click an event and ignore, trash, snooze any line item. Maybe

HA-PC-1 System Uptime changed to 29:16:22:25 2:50:22 PM - 2 minutes ago

HASPone p[2].b[4] displays a clock with a clock icon triggered by time pattern 2:50:00 PM - 2 minutes ago - Traces

issue-triage-workflows[bot] commented 1 year ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

firstdominator commented 1 year ago

I'm up to date on the OS and on the core version and problem still there.

teskanoo commented 1 year ago

Bump. The only reason that hasn't been any activity on this lately is because we're still waiting for the development team to chime in with their diagnosis.

Still ongoing on both web and phone.

chicagoandy commented 1 year ago

Two months later, we still have our logbooks spam'ed with checkins. Does anyone have a recommendation or solution ?

teskanoo commented 1 year ago

Still happening... 2023.07.2

image

None of the logbook entries have a clickable link making me assume they are not 'registered' with an entity_id

If they were (which might allow me to exclude them using logbook exclude rules)

image

but this did not work to remove entities added with no domain or entity_id

exclude:

  domains:
  - logbook # Excludes nothing
  - zha
AndreFjeld commented 1 year ago

Still having this anoying problem. Would really like to get rid of these crap messages from the ZHA integration. Or.. at least be able to filter them out from the logbook.

image

doekwerk commented 1 year ago

Solved months ago: I just uninstalled Matter, ZHA and trashed the Skyconnect and paired all 76 devices to a Hue and Ikea bridge. The Matter devices are sols. Never had this logging issue again. Nor unresponsive devices or any other issue. ZHA, Matter and Skyconnect are still Beta.

chicagoandy commented 1 year ago

Clicking "reconfigure devices" fixed this for me.

issue-triage-workflows[bot] commented 1 year ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

Yardco commented 1 year ago

Issue is still present on 2023.11.0rc

marcel89l commented 1 year ago

Vote for it, that is annoying

geekgarage commented 11 months ago

image Still an issue image

trevorwarwick commented 9 months ago

I'd like to add my vote for getting this addressed somehow. I'm not interested in ANY of the zha internal events (i.e. the ones with the BIG RED Z icon) but as others have mentioned, there doesn't seem to be a supported way of not seeing them in the logbook. Filtering using recorder, or logbook, doesn't seem to catch them.

Screenshot 2024-01-25 145327

What I did find is that you can get rid of them by hacking the code directly, commenting out the last line of components/zha/logbook.py. This doesn't stop them being logged, but it does stop them being displayed, as it makes the event display callback routine essentially do nothing.

`

async_describe_event(ZHA_DOMAIN, ZHA_EVENT, async_describe_zha_event)

`

It seems this issue should be simple to fix for someone who actually understands how these subsystems fit together. It just seems to need some proper classifier added to the events, that can then be filtered. I've yet to work out from the code where the events are actually being fed back and what causes them to be considered for the logbook at all, and that may be beyond my minimal python skills...

dokujah commented 9 months ago

Current logbook behavoir is that when I do a page refresh the ZHA and Hue events which should not be there are gone from the logbook. They are not written to the DB so the recorder ZHA and Hue events exclude works. So only in the live feed. This was discussed last year in another thread leading to #72728

That's not my experience. I still see the entries after restarting HASS so they must come from the database, right?

hifialan commented 8 months ago

Any progress on this? It is driving me crazy. Interesting is that I have 2 of these Smartthings devices, and only one is filling the log with these useless events.

hlowe59 commented 8 months ago

This should be a simple fix (???). What I miss is a response on this thread,, to get an understanding why the log i filled with useless information - and why it is not possible to exclude them in any way. Maybe this is the curse of using open source software - it is more fun to develop new features than fixing small issues for the users? Please help, and I will for certain give the developer(s) a lot of credit!

mukul-k commented 8 months ago

also experiencing same issue, but only with 2 zigbee extenders.

mbo18 commented 8 months ago

I would also like to exclude ZHA events from the logbook. Since 2024.3, I have events every 5min for a plug:

event_type: zha_event
data:
  device_ieee: 60:a4:23:ff:fe:65:1e:6e
  unique_id: 60:a4:23:ff:fe:65:1e:6e:1:0x0019
  device_id: 385fb0ca6c0f2ae492d146c240089cc8
  endpoint_id: 1
  cluster_id: 25
  command: attribute_updated
  args:
    attribute_id: 2
    attribute_name: current_file_version
    value: 65
  params: {}
origin: LOCAL
time_fired: "2024-03-08T10:07:16.788963+00:00"
context:
  id: 01HRERG0SMN27BTTDYHT1FR39M
  parent_id: null
  user_id: null
Capture d’écran 2024-03-08 à 11 18 37

@dmulcahey @puddly Sorry to tag you but this issue is old. Is there anything you can do?

trevorwarwick commented 8 months ago

FWIW, I've managed to remove most of the ZHA stuff in the logbook with these bits of config. There's still the bug where if you're watching the live logbook update the ZHA events will appear, but they then disappear if you refresh the view, but I find this liveable-with.

The "logbook" section removes any events that don't have a recognised filterable domain, so it's a catch-all, and may remove some other events that you do care about...

recorder:
  exclude:
    event_types:
      - zha_event

logbook:
  exclude:
    domains:
      - logbook
goodlucknow commented 7 months ago

+1 on this issue. It's making the logbook pretty hard to use.

luke7101 commented 7 months ago

I don't know if these two symptoms are related but recently (a few months ago I guess) my Xfinity keypad started to act weirdly. I have an automation supposed to sync Alarmo and the keypad and it started to be slow/laggy/buggy for no reason. The automation is triggered when a ZHA event is detected with the keypad, and it started to be triggered with no one touching the keypad. By taking a look at the logs I noticed the "attribute update event" was triggering it, and I'm pretty sure there was no attribute update events at all a few months ago. I guess if it started with the introduction of their new "firmware update" check that runs all day long for every ZHA device.

goodlucknow commented 7 months ago

@luke7101 I've found most common blueprints for ZHA switches are always triggered by these update events also, but the logic within the automation means no action is taken. I'm not sure what kind of overhead this creates but similarly to the logbook it can spam automation traces making it harder to get useful info.

luke7101 commented 7 months ago

@luke7101 I've found most common blueprints for ZHA switches are always triggered by these update events also, but the logic within the automation means no action is taken. I'm not sure what kind of overhead this creates but similarly to the logbook it can spam automation traces making it harder to get useful info.

I absolutely agree. No action from the automation logic, automation traces messed up, all in all a useless spam activity. I just don't understand why this issue is open since 2022, but I just got impacted since core2024 ... Isolating the source of this behavior would help stopping it.

mbo18 commented 7 months ago

Good news, with https://github.com/home-assistant/core/pull/114591 attribute update events due to ZHA checking for updates should disappear

teskanoo commented 7 months ago

I just saw that made beta - fingers crossed we have seen the end of this :)

luke7101 commented 7 months ago

core-2024.4.0 fixed the issue for me. No more flooding in the logbook and no more automations triggered by a fake event!

Yardco commented 7 months ago

still seeing 'Checkin event was fired' events after upgrading unfortunately

issue-triage-workflows[bot] commented 4 months ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

Klausenzweitausen commented 4 months ago

I still have that issue. I'm pretty new to homeassistant and these messages appeared after installing an Sonoff Zigbee 3.0 USB Dongle Plus and connected some IKEA Sensors and Light Core 2024.7.0 Supervisor 2024.06.2 Operating System 12.4 Frontend 20240703.0

teskanoo commented 3 months ago

FWIW... Also exp experiencing this with a Sonoff Zigbee 3.0 USB Dongle Plus

melyux commented 1 month ago

Still happening for Hue motion sensors.

comatose-tortoise commented 3 weeks ago

Yes, I have a Hue outdoor motion sensor that's spamming the logbook on the latest version.

teskanoo commented 1 week ago

IT is mainly Hue motions sensors that still persist in this behaviour for me now