Upgraded HASS to 2024.7 getting lots of error logs

[cweakland]

These look like they may be icloud3 related, I was hoping someone my be able help me resolve this. I did not see these prior to the upgrade. Here is the log in question from my homeassistant.log, I get one every few seconds.

2024-07-03 13:47:04.948 ERROR (MainThread) [homeassistant.core] Error running job: <Job listen HassJobType.Callback <function Recorder.async_initialize.._event_listener at 0x7fc27ff76480>> Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/core.py", line 1559, in async_fire_internal self._hass.async_run_hass_job(job, event) File "/usr/src/homeassistant/homeassistant/core.py", line 938, in async_run_hass_job hassjob.target(args) File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 330, in _event_listener if entity_filter(entity_id): ^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/icloud3/support/recorder_prefilter.py", line 123, in entity_filter return recorder_entity_filter(entity_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: 'NoneType' object is not callable

[kksligh]

same - its literally crashing my instance because of the amount of errors

[cweakland]

same - its literally crashing my instance because of the amount of errors

Yep, I had to roll back to last months release, bummer.

[gcobb321]

I am away so I can not issue an update but try this.

Edit the _config/custom_components/cloud3/support/recorderprefilter.py file. insert the following at line 122 before the line causing the error at line 123.

        if entity_id is None: return False

The code should line up with the if entity_id line on 119. Then restart HA

[kksligh]

unfortunately, I literally had to delete it within 30-60 seconds of booting up or it would crash my instance. Not sure why because I have it on a very powerful NUC. It would lock up and when I looked at the logs, it was throwing errors 3000+ times within seconds of booting up - I had several devices so maybe thats it. I'll wait for an official update before reinstalling - love the work!

[gcobb321]

I’m actually in Switzerland on a trip and do not want to update HA while I’m away. That never ends well. The HA guys must have made some changes in 2024.7 that did something. That module injects code into the HA recorder module to be able to stop logging entity changes for the Event Log, info and other fields.

[kksligh]

here's what someone responded with....

[stickpin]

The problem is recorder_entity_filter itself and not an entity_id.

The temporary fix is actually to add this line: if recorder_entity_filter is None: return False in the config/custom_components/cloud3/support/recorder_prefilter.py file. The code should look like this:

    def entity_filter(entity_id):
        """
        Prefilter an entity to see if it should be excluded from
        the recorder history.

        This function is injected into the recorder, replacing the
        original HA recorder_entity_filter module.

        Return:
            False - The entity should is in the filter list
            Run the original HA recorder_entity_filter function -
                The entity is not in the filter list.
        """
        if (entity_id
                and entity_id in hass.data['recorder_prefilter']['exclude_entities']):
            return False
        if recorder_entity_filter is None: return False
        return recorder_entity_filter(entity_id)

[gcobb321]

I am in Switzerland and cannot release an update to iCloud3 to fix this error. However, I have updated the code in GitHub.

Download the support/recorder_prefilter.py file in the iCloud3 GitHub repository here and save it to the icloud3/support directory. (Look for ‘Raw’ in the menu bar above the code on the right. The Download icon is the 2nd icon to the right) Restart HA.

[gcobb321]

@stickpin Thanks. I just added your code to _recorderprefilter.py

[cweakland]

Thank you both!

[stickpin]

@cweakland i think, it makes sense to reopen this bug. After the change implemented by @gcobb321 your recorder is still broken.

For the time being, I've completely commented _inject_filter in lines 82, 83.

    if hass.data.get('recorder_prefilter') is None:
        rp_data = hass.data['recorder_prefilter'] = {}
        rp_data['injected'] = True
        rp_data['legacy'] = True
        rp_data['exclude_entities'] = []

        try:
            ha_recorder.entity_filter._exclude_e.add(entities)
            return True
        except:
            pass

        rp_data['legacy'] = False

        #if _inject_filter(hass) is False:
        #    return

I will see if I'll have some time today to debug it properly.

[cweakland]

By recorder, do you mean the "logbook"? If so, yes, my logbook is pretty empty.

[cweakland]

@stickpin The workaround you recommended brought my logbook back to life.

[gcobb321]

I have updated the _recorderfilter.py module in the GitHub repositories to comment out the inject filter code suggested very @stickpin

@Snuffy2 Have you been following this?

[Snuffy2]

Nope, but I am now :-) I linked a similar/same issue from another repo where I used recorder_history_prefilter. Interestingly, I'm not seeing the error in my logs and I'm running 3 integrations with it. Will investigate more.

[ahamro]

@gcobb321 wrote:

_Download the support/recorder_prefilter.py file in the iCloud3 GitHub repository here and save it to the icloud3/support directory. (Look for ‘Raw’ in the menu bar above the code on the right. The Download icon is the 2nd icon to the right) Restart HA._

Thanks for the quick solution. It worked like a charm.

Enjoy your trip!

Cheers, -Ton.

[uandco]

Confirming replacing the file and restarting HA fixes the recorder issue.

[thenoid]

homeassistant  | 2024-07-05 16:57:05.946 WARNING (Recorder) [homeassistant.components.recorder.db_schema] State attributes for sensor.icloud3_event_log exceed maximum size of 16384 bytes. This can cause database performance issues; Attributes will not be stored

after applying the patch, this is the new error of fun.

[gcobb321]

Ignore I it. This is a problem a few of us are working on. I’m in Switzerland right now and have not upgraded my R pi to 2024.7 since in away.

[swoop124]

Hello,

i've got the same errors after Update to 2024.7.0 then i rolled back to 2024.6.4 and tried to update to 2024.7.1 but got the same errors:

2024-07-04 13:45:19.970 ERROR (MainThread) [homeassistant.core] Error running job: <Job listen * HassJobType.Callback <function Recorder.async_initialize.<locals>._event_listener at 0x7fa7b5253600>>
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/core.py", line 1559, in async_fire_internal
    self._hass.async_run_hass_job(job, event)
  File "/usr/src/homeassistant/homeassistant/core.py", line 938, in async_run_hass_job
    hassjob.target(*args)
  File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 330, in _event_listener
    if entity_filter(entity_id):
       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/icloud3/support/recorder_prefilter.py", line 123, in entity_filter
    return recorder_entity_filter(entity_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/core.py", line 1559, in async_fire_internal
    self._hass.async_run_hass_job(job, event)
  File "/usr/src/homeassistant/homeassistant/core.py", line 938, in async_run_hass_job
    hassjob.target(*args)
  File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 330, in _event_listener
    if entity_filter(entity_id):
       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/icloud3/support/recorder_prefilter.py", line 123, in entity_filter
    return recorder_entity_filter(entity_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable
2024-07-04 13:45:19.971 ERROR (MainThread) [homeassistant.core] Error running job: <Job listen * HassJobType.Callback <function Recorder.async_initialize.<locals>._event_listener at 0x7fa7b5253600>>
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/core.py", line 1559, in async_fire_internal
    self._hass.async_run_hass_job(job, event)
  File "/usr/src/homeassistant/homeassistant/core.py", line 938, in async_run_hass_job
    hassjob.target(*args)
  File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 330, in _event_listener
    if entity_filter(entity_id):
       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/icloud3/support/recorder_prefilter.py", line 123, in entity_filter
    return recorder_entity_filter(entity_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable
2024-07-04 13:45:19.972 ERROR (MainThread) [homeassistant.core] Error running job: <Job listen * HassJobType.Callback <function Recorder.async_initialize.<locals>._event_listener at 0x7fa7b5253600>>
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/core.py", line 1559, in async_fire_internal
    self._hass.async_run_hass_job(job, event)
  File "/usr/src/homeassistant/homeassistant/core.py", line 938, in async_run_hass_job
    hassjob.target(*args)
  File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 330, in _event_listener
    if entity_filter(entity_id):
       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/icloud3/support/recorder_prefilter.py", line 123, in entity_filter
    return recorder_entity_filter(entity_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable

any news?

[gcobb321]

I am in Switzerland and cannot release an update to iCloud3 to fix this error. I also cannot access my RPi server. However, I have updated the code in GitHub.

1. Download the support/recorder_prefilter.py file in the iCloud3 GitHub repository here and save it to the icloud3/support directory. (Look for ‘Raw’ in the menu bar above the code on the right. The Download icon is the 2nd icon to the right)
2. Restart HA.

This is a temporary fix until some updates are done. You may lose some HA log history until this is resolved. I am involved in some discussions on changes to HA that will handle this issue

other than that, there is no more news.

[smarthomefamilyverrips]

@gcobb321 will this https://github.com/home-assistant/core/pull/121658 mean the whole icloud3 integration stop working if update to HA 2024.7.2? Or it will only stop storing data in recorder?

[u20p17]

yes, if you update to 2024.7.2 the icloud3 integration will not get loaded... i just restored to 2024.7.1 with the temporary fix mentioned above...

[smarthomefamilyverrips]

yes, if you update to 2024.7.2 the icloud3 integration will not get loaded... i just restored to 2024.7.1 with the temporary fix mentioned above...

@u20p17 thanks for your reply, I will just stay on 2024.6.4 then till Gary is back from vacation and had time to apply a permanent fix 👍

[broyuken]

This only affects icloud3 correct? Or does it also break icloud3_v3? I haven’t updated to 2024.7.2 yet and am hesitant to now after seeing those release notes.

[gcobb321]

iCloud3 inserts special code into the HA recorder that automatically excludes some text based iCloud3 sensors from the HA log database. HA release 3024.7 broke this and 2024.7.2 prevented iCloud3 from loading if it tried to do this injection.

I am in Switzerland and cannot release an update to iCloud3 to fix this error. However, I have updated the code in GitHub.

1. Download the support/recorder_prefilter.py file in the iCloud3 GitHub repository here and save it to the icloud3/support directory. (Look for ‘Raw’ in the menu bar above the code on the right. The Download icon is the 2nd icon to the right)
2. Restart HA.

The Event Log also adds a large item into the log and will generate a warning message in the Error Log. To stop the warning message and exclude this sensor from being added, add the following to the configuration.yaml file. It adds a manual exclude for sensor.

recorder:
  exclude:
    entities:
      - sensor.icloud3_event_log

[smarthomefamilyverrips]

This only affects icloud3 correct? Or does it also break icloud3_v3? I haven’t updated to 2024.7.2 yet and am hesitant to now after seeing those release notes.

From how I understand it affects as well iCloud3 as iCloud3_V3 anyone correct me if I am wrong

[gcobb321]

Both iCloud3 and iCloud3 _v3

[broyuken]

Got it, I'll hold off on updating then until you get back. No rush, but when are you back just so we have a rough idea?

[gcobb321]

In 3-days. I should have an update out by 7/15. It will disable the HA special patch with instructions until the issue is resolved.

[thenoid]

Sounds like we need to get Santa (Gary) some elves to release things when he's taking a break.

[gcobb321]

My next trip is Lisbon to Cape Town on the west coast of Africa from 11/10 to 12/7. Hopefully the HA guys won’t break iCloud3 on the 2024.12 update.

[broyuken]

Must be nice to be retired :)

[nebulous]

Adding backreference to the hass community side of this:

https://community.home-assistant.io/t/psa-2024-7-recorder-problems/746428

[gcobb321]

iCloud3 v3.0.5.5 (7/16/2024) has been released that fixes this problem. It should be available on HACS in a day or so. No other changes have been made.

[JoTec2002]

After the update to the new version v3.0.5.5 and the HA update to 2024.7.2 the icloud3 integration doesnt load. Possible related to https://github.com/home-assistant/core/pull/121658 ?

[gcobb321]

HACS was the problem. It was downloading the v3.0.5.2 release, not the v3.0.5.5 release even though the zip file for the release was correct and the files were correct. I will create a new release and see what happens.

Here is the v3.0.5.5 zip file. icloud3.zip

[gcobb321]

It was my error. I did everything except update the _customcomponents/icloud3 code files on the GitHub repository so it was picking up v3.0.5.2 code instead.

All is good now. HACS updates the files with the v3.0.5.5 version after refreshing via the Update Information option.

[gcobb321]

Completed with iCloud3 v3.0.5.5/v3.0.5.6