Detected blocking call to dsmr_parser

MaVeWeb commented 3 months ago

The problem

After updating to HA 2024.6.0, I get 597 times the warning:

Detected blocking call to open with args ('/usr/local/lib/python3.12/site-packages/pytz/zoneinfo/Universal', 'rb') in /usr/local/lib/python3.12/site-packages/pytz/init.py, line 108: return open(filename, 'rb') inside the event loop Traceback

What version of Home Assistant Core has the issue?

core-2024.6.0

What was the last working version of Home Assistant Core?

core-2024.5.5

What type of installation are you running?

Home Assistant OS

Integration causing the issue

May be dsmr_parser ????

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

2024-06-05 22:53:20.785 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to open with args ('/usr/local/lib/python3.12/site-packages/pytz/zoneinfo/Africa/Abidjan', 'rb') in /usr/local/lib/python3.12/site-packages/pytz/__init__.py, line 108: return open(filename, 'rb') inside the event loop
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module>
    sys.exit(main())
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main
    exit_code = runner.run(runtime_conf)
  File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 672, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 639, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1988, in _run_once
    handle._run()
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/local/lib/python3.12/site-packages/serial_asyncio/__init__.py", line 120, in _read_ready
    self._protocol.data_received(data)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/clients/protocol.py", line 133, in data_received
    self.handle_telegram(telegram)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/clients/protocol.py", line 159, in handle_telegram
    parsed_telegram = self.telegram_parser.parse(telegram)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 95, in parse
    dsmr_object = object["value_parser"].parse(match)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 311, in parse
    values=self._parse(line)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 219, in _parse
    return self._parse_values(values)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 190, in _parse_values
    return [self.value_formats[i].parse(value)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 399, in parse
    value = self.coerce_type(value) if value is not None else value
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/value_types.py", line 24, in timestamp
    local_tz = pytz.timezone('Europe/Amsterdam')
  File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 181, in timezone
    if zone in all_timezones_set:  # noqa
  File "/usr/local/lib/python3.12/site-packages/pytz/lazy.py", line 154, in _lazy
    for i in fill_iter.pop():
  File "/usr/local/lib/python3.12/site-packages/pytz/lazy.py", line 101, in _lazy
    list.extend(self, fill_iter.pop())
  File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 1115, in <genexpr>
    tz for tz in _all_timezones_unchecked if resource_exists(tz))
  File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 121, in resource_exists
    open_resource(name).close()
  File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 108, in open_resource
    return open(filename, 'rb')

2024-06-05 22:53:20.789 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to open with args ('/usr/local/lib/python3.12/site-packages/pytz/zoneinfo/Africa/Accra', 'rb') in /usr/local/lib/python3.12/site-packages/pytz/__init__.py, line 108: return open(filename, 'rb') inside the event loop
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module>
    sys.exit(main())
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main
    exit_code = runner.run(runtime_conf)
  File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 672, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 639, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1988, in _run_once
    handle._run()
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/local/lib/python3.12/site-packages/serial_asyncio/__init__.py", line 120, in _read_ready
    self._protocol.data_received(data)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/clients/protocol.py", line 133, in data_received
    self.handle_telegram(telegram)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/clients/protocol.py", line 159, in handle_telegram
    parsed_telegram = self.telegram_parser.parse(telegram)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 95, in parse
    dsmr_object = object["value_parser"].parse(match)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 311, in parse
    values=self._parse(line)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 219, in _parse
    return self._parse_values(values)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 190, in _parse_values
    return [self.value_formats[i].parse(value)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 399, in parse
    value = self.coerce_type(value) if value is not None else value
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/value_types.py", line 24, in timestamp
    local_tz = pytz.timezone('Europe/Amsterdam')
  File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 181, in timezone
    if zone in all_timezones_set:  # noqa
  File "/usr/local/lib/python3.12/site-packages/pytz/lazy.py", line 154, in _lazy
    for i in fill_iter.pop():
  File "/usr/local/lib/python3.12/site-packages/pytz/lazy.py", line 101, in _lazy
    list.extend(self, fill_iter.pop())
  File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 1115, in <genexpr>
    tz for tz in _all_timezones_unchecked if resource_exists(tz))
  File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 121, in resource_exists
    open_resource(name).close()
  File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 108, in open_resource
    return open(filename, 'rb')

Additional information

No response

peetersch commented 3 months ago

I have the same issue. Returning to HA version 2024.5.5 fixed the issue.

bdraco commented 3 months ago

I have the same issue.

Returning to HA version 2024.5.5 fixed the issue.

2024.5.5 isn't capable of detecting this problem so it's still there but blissfully unaware

bdraco commented 3 months ago

https://developers.home-assistant.io/blog/2024/05/19/fix_zoneinfo_blocking_io/

elupus commented 3 months ago

This is fixed upstream https://github.com/ndokter/dsmr_parser/pull/157

Needs a library bump

home-assistant[bot] commented 3 months ago

Hey there @robbie1221, @frenck, mind taking a look at this issue as it has been labeled with an integration (dsmr) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `dsmr` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign dsmr` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

dsmr documentation dsmr source _{^{(message by IssueLinks)}}

bdraco commented 3 months ago

I think this is fixed upstream but the lib needs to be bumped

bdraco commented 3 months ago

Looks like there some tests that will need to be adjusted

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ERROR collecting tests/components/dsmr/test_mbus_migration.py ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
ImportError while importing test module '/Users/bdraco/home-assistant/tests/components/dsmr/test_mbus_migration.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/components/dsmr/test_mbus_migration.py:6: in <module>
    from dsmr_parser.obis_references import (
E   ImportError: cannot import name 'BELGIUM_MBUS1_DEVICE_TYPE' from 'dsmr_parser.obis_references' (/Users/bdraco/home-assistant/venv/lib/python3.12/site-packages/dsmr_parser/obis_references.py)

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ERROR collecting tests/components/dsmr/test_sensor.py ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
ImportError while importing test module '/Users/bdraco/home-assistant/tests/components/dsmr/test_sensor.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/components/dsmr/test_sensor.py:14: in <module>
    from dsmr_parser.obis_references import (
E   ImportError: cannot import name 'BELGIUM_MBUS1_DEVICE_TYPE' from 'dsmr_parser.obis_references' (/Users/bdraco/home-assistant/venv/lib/python3.12/site-packages/dsmr_parser/obis_references.py)
collected 49 items / 2 errors

Sadly I don't have the ability to test this one so it will need to wait for someone who can do the changes

OhioSpyderman commented 1 month ago

Still receiving this error with latest core update (yesterday). Not sure if it is related (probably not) but I also cannot get info from the ABOUT page to fill in supervisor, core...etc.

Detected blocking call to open with args ('/config/.ps4-games.0CFE45D4C804_639d.json',) inside the event loop by integration 'ps4' at homeassistant/components/ps4/init.py, line 176: games = load_json_object(g_file) (offender: /usr/src/homeassistant/homeassistant/util/json.py, line 74: with open(filename, mode="rb") as fdesc:), please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue+label%3A%22integration%3A+ps4%22 For developers, please see https://developers.home-assistant.io/docs/asyncio_blocking_operations/#open Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/usr/src/homeassistant/homeassistant/main.py", line 223, in sys.exit(main()) File "/usr/src/homeassistant/homeassistant/main.py", line 209, in main exit_code = runner.run(runtime_conf) File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run return loop.run_until_complete(setup_and_run_hass(runtime_config)) File "/usr/local/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete self.run_forever() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 641, in run_forever self._run_once() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1990, in _run_once handle._run() File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 727, in _async_add_entity await entity.async_device_update(warning=False) File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1300, in async_device_update await self.async_update() File "/usr/src/homeassistant/homeassistant/components/ps4/media_player.py", line 160, in async_update self._parse_status() File "/usr/src/homeassistant/homeassistant/components/ps4/media_player.py", line 166, in _parse_status self._games = load_games(self.hass, cast(str, self.unique_id)) File "/usr/src/homeassistant/homeassistant/components/ps4/init.py", line 176, in load_games games = load_json_object(g_file)

Bob.

elupus commented 1 month ago

@OhioSpyderman please open a new issue. Your issue is on the ps4 integration.

home-assistant / core