search

custom-components / pyscript

Pyscript adds rich Python scripting to HASS
Apache License 2.0
873 stars 46 forks source link

Detected blocking call to scandir with args (b'/config/pyscript',) #616

Closed psykokwak-com closed 2 months ago

psykokwak-com commented 2 months ago

Since last HA update, I have the following error in loop :

2024-07-08 11:06:49.641 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to scandir with args (b'/config/pyscript',) inside the event loop by custom integration 'pyscript' at custom_components/pyscript/init.py, line 335: observer.start() (offender: , line 366: ?), please create a bug report at https://github.com/custom-components/pyscript/issues For developers, please see https://developers.home-assistant.io/docs/asyncio_blocking_operations/#scandir Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/usr/src/homeassistant/homeassistant/main.py", line 223, in sys.exit(main()) File "/usr/src/homeassistant/homeassistant/main.py", line 209, in main exit_code = runner.run(runtime_conf) File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run return loop.run_until_complete(setup_and_run_hass(runtime_config)) File "/usr/local/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete self.run_forever() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 641, in run_forever self._run_once() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1990, in _run_once handle._run() File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) File "/usr/src/homeassistant/homeassistant/runner.py", line 166, in setup_and_run_hass return await hass.async_run() File "/usr/src/homeassistant/homeassistant/core.py", line 523, in async_run await self.async_start() File "/usr/src/homeassistant/homeassistant/core.py", line 576, in async_start self.bus.async_fire_internal(EVENT_HOMEASSISTANT_STARTED) File "/usr/src/homeassistant/homeassistant/core.py", line 1559, in async_fire_internal self._hass.async_run_hass_job(job, event) File "/usr/src/homeassistant/homeassistant/core.py", line 941, in async_run_hass_job return self._async_add_hass_job(hassjob, args, background=background) File "/usr/src/homeassistant/homeassistant/core.py", line 756, in _async_add_hass_job task = create_eagertask( File "/usr/src/homeassistant/homeassistant/util/async.py", line 37, in create_eager_task return Task(coro, loop=loop, name=name, eager_start=True) File "/config/custom_components/pyscript/init.py", line 335, in hass_started observer.start()

My HA informations :

System Information

version core-2024.7.1
installation_type Home Assistant OS
dev false
hassio true
docker true
user root
virtualenv false
python_version 3.12.4
os_name Linux
os_version 6.6.33-haos
arch x86_64
timezone Europe/Paris
config_dir /config
Home Assistant Community Store GitHub API | ok -- | -- GitHub Content | ok GitHub Web | ok GitHub API Calls Remaining | 4785 Installed Version | 1.34.0 Stage | running Available Repositories | 1394 Downloaded Repositories | 10
Home Assistant Cloud logged_in | false -- | -- can_reach_cert_server | ok can_reach_cloud_auth | ok can_reach_cloud | ok
Home Assistant Supervisor host_os | Home Assistant OS 12.4 -- | -- update_channel | stable supervisor_version | supervisor-2024.06.2 agent_version | 1.6.0 docker_version | 26.1.4 disk_total | 503.3 GB disk_used | 169.2 GB healthy | true supported | true host_connectivity | true supervisor_connectivity | true ntp_synchronized | true virtualization | kvm board | ova supervisor_api | ok version_api | ok installed_addons | Terminal & SSH (9.14.0), File editor (5.8.0), Samba share (12.3.1), Studio Code Server (5.15.0), SQLite Web (4.1.2)
Dashboards dashboards | 2 -- | -- resources | 7 views | 9 mode | storage
Recorder oldest_recorder_run | 8 juillet 2024 à 09:06 -- | -- current_recorder_run | 8 juillet 2024 à 11:06 estimated_db_size | 3.54 MiB database_engine | sqlite database_version | 3.45.3
Xiaomi Miot Auto component_version | 0.7.18 -- | -- can_reach_server | ok can_reach_spec | ok logged_accounts | 1 total_devices | 4
craigbarratt commented 2 months ago

This is a repeat of #612. It should be fixed with commit 27ca121. It would be great if you could test the latest code in github to confirm it's fixed (and not otherwise broken).

psykokwak-com commented 2 months ago

Thanks for your answer. I just updated the latest code from github and I still have the issue : 

2024-07-08 13:05:59.927 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to scandir with args (b'/config/pyscript',) inside the event loop by custom integration 'pyscript' at custom_components/pyscript/__init__.py, line 340: observer.start() (offender: <frozen os>, line 366: ?), please create a bug report at https://github.com/custom-components/pyscript/issues
For developers, please see https://developers.home-assistant.io/docs/asyncio_blocking_operations/#scandir
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module>
    sys.exit(main())
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main
    exit_code = runner.run(runtime_conf)
  File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 641, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1990, in _run_once
    handle._run()
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/config/custom_components/pyscript/__init__.py", line 340, in hass_started
    observer.start()
craigbarratt commented 2 months ago

Thanks for checking,. My fix didn't work. I'm traveling tomorrow and I'll try to get to it on the flight (assuming the WiFi is decent).

ProphetOfDoom commented 2 months ago

Just to let you know I'm getting the same warning here. I know it's affected a number of other integrations (Such as the Solcast Solar PV integration https://github.com/BJReplay/ha-solcast-solar). If you can stick the changes up as a beta release I'll give it a whirl. There was some discussion over on that repo (new management!) about how to do that if you aren't sure.

craigbarratt commented 2 months ago

Ok, I pushed another potential fix 774842d. Just about to depart on a flight, so I haven't tested it much....

psykokwak-com commented 2 months ago

Thanks, Seems good this time 👍

ProphetOfDoom commented 2 months ago

Just updated my HA instance with code from pyscript/custom_components/pyscript, restarted HA and run my automation that uses PyScript. No PyScript Warnings/Errors in HA system log Thought you might like to know.

craigbarratt commented 2 months ago

Thanks for checking and confirming the fix!

Joao-Sousa-71 commented 2 months ago

@craigbarratt I've updated my HA instance and the warning disappeared for all my pyscripts except one:

2024-07-09 19:36:35.106 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to open with args ('/config/multiscrape/solzaima_ha_integration/page_response_headers.txt', 'rb') inside the event loop by custom integration 'pyscript' at custom_components/pyscript/eval.py, line 1982: return func(*args, **kwargs) (offender: /usr/local/lib/python3.12/shutil.py, line 260: with open(src, 'rb') as fsrc:), please create a bug report at https://github.com/custom-components/pyscript/issues
For developers, please see https://developers.home-assistant.io/docs/asyncio_blocking_operations/#open
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module>
    sys.exit(main())
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main
    exit_code = runner.run(runtime_conf)
  File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 641, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1990, in _run_once
    handle._run()
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/config/custom_components/pyscript/function.py", line 448, in run_coro
    result = await coro
  File "/config/custom_components/pyscript/eval.py", line 508, in do_service_call
    retval = await func.call(ast_ctx, **data)
  File "/config/custom_components/pyscript/eval.py", line 800, in call
    val = await self.try_aeval(ast_ctx, arg1)
  File "/config/custom_components/pyscript/eval.py", line 700, in try_aeval
    return await ast_ctx.aeval(arg)
  File "/config/custom_components/pyscript/eval.py", line 940, in aeval
    val = await getattr(self, name, self.ast_not_implemented)(arg)
  File "/config/custom_components/pyscript/eval.py", line 1029, in ast_if
    val = await self.aeval(arg1)
  File "/config/custom_components/pyscript/eval.py", line 940, in aeval
    val = await getattr(self, name, self.ast_not_implemented)(arg)
  File "/config/custom_components/pyscript/eval.py", line 1367, in ast_expr
    return await self.aeval(arg.value)
  File "/config/custom_components/pyscript/eval.py", line 940, in aeval
    val = await getattr(self, name, self.ast_not_implemented)(arg)
  File "/config/custom_components/pyscript/eval.py", line 1937, in ast_call
    return await self.call_func(func, func_name, *args, **kwargs)
  File "/config/custom_components/pyscript/eval.py", line 1982, in call_func
    return func(*args, **kwargs)

Now, I dont' if it is an issue of the script itself... but it was not showing any warning before the HA upgrade: Here is the script:

@service
def copy_myceza_response_headers():
    """Python Script to copy page_response_headers.txt to the sensor folder file"""
    import os.path
    import shutil
    source_folder = "/config/multiscrape/solzaima_ha_integration/page_response_headers.txt" 
    target_folder = "/config/multiscrape/tokens/page_response_headers.txt"
    check_file = os.path.isfile(source_folder)
    if check_file:
        shutil.copyfile(source_folder, target_folder)
        log.info(f"Python Script to copy page_response_headers.txt to the sensor folder file finished!!!")
    else:
        log.info(f"page_response_headers.txt does not exist in source folder!!!")

Thank you!

craigbarratt commented 2 months ago

Your code is doing file system operations (os.path.isfile and shutil.copyfile), which are considered blocking operations. You could solve this by making this function run in its own thread with the @pyscript_executor decorator (note that log.info() won't work inside a function with that decorator). See the docs.

Joao-Sousa-71 commented 2 months ago

I’ll try to make the changes and check how it goes, I,m not sure if my Python knowledge is enough for this. Thank you for your feedback.