Failed to load automations on startup

[nadir-lalla]

Describe the bug Add-on does not load at all. nothing is shown on screen except for an error: Failed to load automations from /config ('near "OFF": syntax error',)

To Reproduce Steps to reproduce the behavior:

1. Run shortumation

Expected behavior Expected to see the shortumation UI with all automations

Screenshots

Desktop (please complete the following information):

• OS: Windows
• Browser Chrome
• Version: 104.0.5112.102 (Official Build) (64-bit)

HA Hardware:

• Home Assistant OS
• Raspberry Pi 4 4GB with external SSD

Additional context

• I have used the shortumation UI previously, it was after one of the updates of either the add-on or the OS updates that the problem started
• I have tried restarting the add-on
• I have tried restarting the HA hardware
• I have tried uninstalling the add-on and installing again

Shortumation Logs

>  /app /app
> --------------------
> BUILD_VERSION = v0.6.3
> HOST = 0.0.0.0:8000
> HASSIO_WS = 
> Python 3.10.6
> --------------------
> [2022-08-31 12:05:38 +0200] [8] [INFO] Running on http://0.0.0.0:8000 (CTRL + C to quit)
> 2022-08-31:12:05:38|INFO[/usr/local/lib/python3.10/site-packages/hypercorn/logging.py:92@info] Running on http://0.0.0.0:8000 (CTRL + C to quit)
> 2022-08-31:12:07:02|INFO[/app/src/automations/loader.py:73@extract_automation_root_refs] Loading automations from 'automation'
> 2022-08-31:12:07:03|WARNING[/app/src/automations/manager.py:69@reload] near "OFF": syntax error
> 2022-08-31:12:07:11|INFO[/app/src/api/routes/socket.py:30@wait_for_data] Websocket closed.
> 2022-08-31:12:07:11|INFO[/app/src/ws_redirect/__init__.py:52@start] Closing socket...
> 2022-08-31:12:07:11|INFO[/app/src/ws_redirect/__init__.py:54@start] Closed socket
> 2022-08-31:12:08:10|INFO[/app/src/automations/loader.py:73@extract_automation_root_refs] Loading automations from 'automation'
> 2022-08-31:12:08:11|WARNING[/app/src/automations/manager.py:69@reload] near "OFF": syntax error
> 2022-08-31:12:12:56|INFO[/app/src/api/routes/socket.py:30@wait_for_data] Websocket closed.
> 2022-08-31:12:12:56|INFO[/app/src/ws_redirect/__init__.py:52@start] Closing socket...
> 2022-08-31:12:12:56|INFO[/app/src/ws_redirect/__init__.py:54@start] Closed socket

[sbcrumb]

I have this very same issue as well. I was about to open a report till I saw this one.

[asosnovsky]

I am assuming your automations are all in one file?

Do you mind sharing your configuratio.yaml (just the parts to do with automations/packages) or anything else that could help me replicate the issue on my end

[nadir-lalla]

This is the base of my cofig.yaml, The rest is not relevant (duckdns, telegram, http etc)

default_config:

group: !include groups.yaml
automation: !include automations.yaml
script: !include scripts.yaml
scene: !include scenes.yaml

frontend:
  themes: !include_dir_merge_named themes
  extra_module_url:
    - /local/card-mod.js
tts:
  - platform: google_translate

sonoff:
  username: [removed]
  password: !secret sonoff
  force_update: [temperature, power]
  scan_interval: '00:02:00' 

homeassistant:
  customize: !include customize.yaml
  packages: !include_dir_named packages

All of my automations were done using the Hass UI. All are located in the automations.yaml file

The most recent change I've done was installed mushroom cards. Thats the only change outside of the usual updates

[nadir-lalla]

@sbcrumb can you provide some info on changes you made before the add-on started to give this issue for you?

Maybe that can help get it sorted

[sbcrumb]

I will do my best. Home Assistant 2022.8.7 Supervisor 2022.08.5 Operating System 8.5 Frontend 20220802.0 - latest

I don't have many unique things. I had installed and fiddled around with the prev versions. I had the scrolling bug that was being worked on so I was just installing updates till that was sorted out. I hadn't got to dive into it. BUt all my automations are in a single file at this time. I am willing to try anything you might need as a test.

# Loads default set of integrations. Do not remove.
default_config:
  - name: redacted
    latitude: redacted
    longitude: redacted
    radius: 50
    icon: mdi:account-multiple
#MariaDB Config
recorder:
  db_url: mysql://homeassistant:redacted@core-mariadb/homeassistant?charset=utf8mb4
homeassistant:
  packages: !include_dir_named packages/    
weather:
  - platform: pirateweather
    api_key: "redacted"
    # Additional optional values:
    latitude: redacted 
    longitude: redacted
    mode: "hourly"
    name:  Weather
notify:
  - name: Pushover_Android
    platform: pushover
    api_key: redacted
    user_key: redacted
  - name: Agent
    platform: hass_agent_notifier
    resource: http://192.168.253.100:5115/notify
  - name: Agent
    platform: hass_agent_notifier
    resource: http://192.168.253.102:5115/notify
#logger:
#  default: warning
#  logs:
#    custom_components.hass_agent_notifier: debug       
# Text to speech
tts:
  - platform: google_translate

#Split Configs
group: !include groups.yaml
#light: !include lights.yaml
#shell_command: !include shell_commands.yaml
sensor: !include sensor.yaml
automation: !include automations.yaml
script: !include scripts.yaml
scene: !include scenes.yaml

frontend:
  themes: !include_dir_merge_named themes

http:
  # For extra security set this to only accept connections on localhost if NGINX is on the same machine
  # Uncommenting this will mean that you can only reach Home Assistant using the proxy, not directly via IP from other clients.
  # server_host: 127.0.0.1
  use_x_forwarded_for: true
  # You must set the trusted proxy IP address so that Home Assistant will properly accept connections
  # Set this to your NGINX machine IP, or localhost if hosted on the same machine.
  trusted_proxies:
    - 192.168.255.0/24
    - 172.19.0.0/24
    - 172.17.0.0/24            

rest_command:
  wake_computer1:
    url: 'http://192.168.253.1/api/wol/wol/set'
    method: POST
    payload: '{"wake":{"interface": "lan","mac": "3C:7C:3F:EF:37:0A"}}'
    content_type: 'application/json'
    username: redacted
    password: redacted
  wake_computer2:
    url: 'http://192.168.253.1/api/wol/wol/set'
    method: POST
    payload: '{"wake":{"interface": "lan","mac": "48:2a:e3:ff:52:1a"}}'
    content_type: 'application/json'
    username: redacted
    password: redacted    

[asosnovsky]

hey guys please give version v0.7.0 a try

also if you still encounter issues please re-post the logs :)

[nadir-lalla]

@asosnovsky V0.7.0 fixed the issue for me. Thanks

[sbcrumb]

Same errors for me with the update.. I will try to get some logs today to you.

[sbcrumb]

Hopefully this helps.

 File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 670, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 266, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 65, in app
    response = await func(request)
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 227, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 162, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/usr/local/lib/python3.10/site-packages/starlette/concurrency.py", line 41, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/app/src/api/routes/automations.py", line 18, in list_autos
    automations.reload()
  File "/app/src/automations/manager.py", line 45, in reload
    batch.append(next(auto_it))
  File "/app/src/automations/loader.py", line 37, in load_and_iter_automations
    for ref in extract_automation_refs(hass_config):
  File "/app/src/automations/loader.py", line 59, in extract_automation_refs
    for ref in chain(
  File "/app/src/automations/loader.py", line 98, in extract_automation_package_refs
    packages = packages_config.to_normalized_json(use_path_as_keys=True)
  File "/app/src/yaml_serializer/__init__.py", line 119, in to_normalized_json
    for path, data in load_dir_yaml(self.path):
  File "/app/src/yaml_serializer/__init__.py", line 17, in load_dir_yaml
    with folder_or_file.open("r") as fp:
  File "/usr/local/lib/python3.10/pathlib.py", line 1119, in open
    return self._accessor.open(self, mode, buffering, encoding, errors,
FileNotFoundError: [Errno 2] No such file or directory: '/config/packages'
2022-09-01:08:46:30|ERROR[/usr/local/lib/python3.10/site-packages/hypercorn/logging.py:100@exception] Error in ASGI Framework
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/hypercorn/asyncio/task_group.py", line 21, in _handle
    await invoke_asgi(app, scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/hypercorn/utils.py", line 247, in invoke_asgi
    await app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 269, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 92, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 147, in simple_response
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/exceptions.py", line 93, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/exceptions.py", line 82, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 670, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 266, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 65, in app
    response = await func(request)
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 227, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 162, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/usr/local/lib/python3.10/site-packages/starlette/concurrency.py", line 41, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/app/src/api/routes/automations.py", line 18, in list_autos
    automations.reload()
  File "/app/src/automations/manager.py", line 45, in reload
    batch.append(next(auto_it))
  File "/app/src/automations/loader.py", line 37, in load_and_iter_automations
    for ref in extract_automation_refs(hass_config):
  File "/app/src/automations/loader.py", line 59, in extract_automation_refs
    for ref in chain(
  File "/app/src/automations/loader.py", line 98, in extract_automation_package_refs
    packages = packages_config.to_normalized_json(use_path_as_keys=True)
  File "/app/src/yaml_serializer/__init__.py", line 119, in to_normalized_json
    for path, data in load_dir_yaml(self.path):
  File "/app/src/yaml_serializer/__init__.py", line 17, in load_dir_yaml
    with folder_or_file.open("r") as fp:
  File "/usr/local/lib/python3.10/pathlib.py", line 1119, in open
    return self._accessor.open(self, mode, buffering, encoding, errors,
FileNotFoundError: [Errno 2] No such file or directory: '/config/packages'

[asosnovsky]

Hopefully this helps.

 File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 670, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 266, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 65, in app
    response = await func(request)
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 227, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 162, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/usr/local/lib/python3.10/site-packages/starlette/concurrency.py", line 41, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/app/src/api/routes/automations.py", line 18, in list_autos
    automations.reload()
  File "/app/src/automations/manager.py", line 45, in reload
    batch.append(next(auto_it))
  File "/app/src/automations/loader.py", line 37, in load_and_iter_automations
    for ref in extract_automation_refs(hass_config):
  File "/app/src/automations/loader.py", line 59, in extract_automation_refs
    for ref in chain(
  File "/app/src/automations/loader.py", line 98, in extract_automation_package_refs
    packages = packages_config.to_normalized_json(use_path_as_keys=True)
  File "/app/src/yaml_serializer/__init__.py", line 119, in to_normalized_json
    for path, data in load_dir_yaml(self.path):
  File "/app/src/yaml_serializer/__init__.py", line 17, in load_dir_yaml
    with folder_or_file.open("r") as fp:
  File "/usr/local/lib/python3.10/pathlib.py", line 1119, in open
    return self._accessor.open(self, mode, buffering, encoding, errors,
FileNotFoundError: [Errno 2] No such file or directory: '/config/packages'
2022-09-01:08:46:30|ERROR[/usr/local/lib/python3.10/site-packages/hypercorn/logging.py:100@exception] Error in ASGI Framework
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/hypercorn/asyncio/task_group.py", line 21, in _handle
    await invoke_asgi(app, scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/hypercorn/utils.py", line 247, in invoke_asgi
    await app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 269, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 92, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 147, in simple_response
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/exceptions.py", line 93, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/exceptions.py", line 82, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 670, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 266, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 65, in app
    response = await func(request)
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 227, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 162, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/usr/local/lib/python3.10/site-packages/starlette/concurrency.py", line 41, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/app/src/api/routes/automations.py", line 18, in list_autos
    automations.reload()
  File "/app/src/automations/manager.py", line 45, in reload
    batch.append(next(auto_it))
  File "/app/src/automations/loader.py", line 37, in load_and_iter_automations
    for ref in extract_automation_refs(hass_config):
  File "/app/src/automations/loader.py", line 59, in extract_automation_refs
    for ref in chain(
  File "/app/src/automations/loader.py", line 98, in extract_automation_package_refs
    packages = packages_config.to_normalized_json(use_path_as_keys=True)
  File "/app/src/yaml_serializer/__init__.py", line 119, in to_normalized_json
    for path, data in load_dir_yaml(self.path):
  File "/app/src/yaml_serializer/__init__.py", line 17, in load_dir_yaml
    with folder_or_file.open("r") as fp:
  File "/usr/local/lib/python3.10/pathlib.py", line 1119, in open
    return self._accessor.open(self, mode, buffering, encoding, errors,
FileNotFoundError: [Errno 2] No such file or directory: '/config/packages'

@sbcrumb ok so your error seems different than @MidnightZA

can you double check that your setup works in HA? it seems that it's unable to find a 'packages' folder in your config. FileNotFoundError: [Errno 2] No such file or directory: '/config/packages'

Also do you mind reporting this in a separate issue since it seems unrelated to this one?

[sbcrumb]

Ok not sure why I don't have config/packages so I removed shortumation, reinstalled added that dir and now it loads. (dir is empty) but at least it works.

[asosnovsky]

@sbcrumb so this folder didn't exists before? why do you have it in the yaml?

good to hear that things work though :)

[sbcrumb]

All very good questions. Could be old stuff from my old install as I recently movd off a Pi to a NUC.