Running docker: "failed to find this automation in '/config' folder" (for packages, split config)

[ghost]

Describe the bug Shortumation picks up the automation titles that are in packages or using !include directory, but I get the error: "failed to find this automation in '/config' folder"

To Reproduce Steps to reproduce the behavior:

1. run Shortumation.

Expected behavior I expect to not get this error

Screenshots

Desktop (please complete the following information):

• OS: MacOS 12.5.1 (but HA running on a Pi-4)
• Browser Safari
• Version 15.6.1

Additional context It works fine for automations created using the UI (using /config/automations.yaml)

[asosnovsky]

what version of Shortumation you are running? it seems like you have an old build (see bottom right of the UI)

[ghost]

I have v 0.6.3

[asosnovsky]

strange, do you mind sharing your configuratio.yaml (just the parts to do with automations/packages) or anything else that could help me replicate the issue on my end

[ghost]

of course, no problem:

packages: !include_dir_named JCHH/packages/
automation: !include automations.yaml
automation JCH: !include_dir_merge_list JCHH/automations/

I'm not honestly sure what else will help. I am running HA in Docker on DietPi (A Lightweight Debian OS).

[asosnovsky]

@jchhenderson please try to update!

also if you still encounter issues please re-post the logs :)

sorry that this issue keeps draging on...

[ghost]

sorry that this issue keeps draging on...

Hey, no worries - you are putting in a lot of work here to provide free software and I appreciate that :-) Updating now and will let you know...

---

Edit: nope, still not working.

[asosnovsky]

@jchhenderson wait I just realized. Are you running shortumation in a separate docker container not addon mode?

If yea, can you do the following:

• Check that the tag is mapped to the latest version v0.7.0
• if using compose do docker compose pull && docker compose build --no-cache
• if using just docker docker pull asosnovsky/shortumation:v0.7.0

I'm starting to think you are experiencing a caching issue in docker.

[Sholofly]

I'm having the same issue. Running v7.0 on latest beta 2022.9.0b1. Installed as Add On. I have my automation integration as a package: folder/integration.yaml in which I include the main integrations.yaml from the root of my config folder. Logging says:

2022-09-01:15:28:31|ERROR[/usr/local/lib/python3.10/site-packages/hypercorn/logging.py:100@exception] Error in ASGI Framework
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/hypercorn/asyncio/task_group.py", line 21, in _handle
    await invoke_asgi(app, scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/hypercorn/utils.py", line 247, in invoke_asgi
    await app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 269, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 92, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 147, in simple_response
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/exceptions.py", line 93, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/exceptions.py", line 82, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 670, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 266, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 65, in app
    response = await func(request)
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 227, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 162, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/usr/local/lib/python3.10/site-packages/starlette/concurrency.py", line 41, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/app/src/api/routes/automations.py", line 18, in list_autos
    automations.reload()
  File "/app/src/automations/manager.py", line 45, in reload
    batch.append(next(auto_it))
  File "/app/src/automations/loader.py", line 37, in load_and_iter_automations
    for ref in extract_automation_refs(hass_config):
  File "/app/src/automations/loader.py", line 59, in extract_automation_refs
    for ref in chain(
  File "/app/src/automations/loader.py", line 100, in extract_automation_package_refs
    yield from extract_automation_inline_package_refs(
  File "/app/src/automations/loader.py", line 172, in extract_automation_inline_package_refs
    raise AssertionError(
AssertionError: configurations.homeassistant.packages[media_player] must be a dictionary!

[ghost]

@jchhenderson wait I just realized. Are you running shortumation in a separate docker container not addon mode?

yes, I am running in a seperate docker container as I am running HA in docker as well. (not HA OS or supervisor).

• Check that the tag is mapped to the latest version v0.7.0

the tag in docker-compose? yes, it's 0.7.0 image: asosnovsky/shortumation-armv7:v0.7.0

• if using compose do docker compose pull && docker compose build --no-cache

made no difference, sorry.

---

edit: My full docker-compose (for Shortumation) is:

  shortumation: # http://dietpi:8000/web
    container_name: shortumation
    image: asosnovsky/shortumation-armv7:v0.7.0
    ports:
      - 8000:8000
    extra_hosts:
      - "host.docker.internal:host-gateway"
    volumes:
      - ./homeassistant:/config       # path is same as Homeassistant
    environment:
      - SUPERVISOR_TOKEN=<renoved>
      - HASSIO_WS=ws://host.docker.internal:8123/api/websocket

[asosnovsky]

@Sholofly

I'm having the same issue. Running v7.0 on latest beta 2022.9.0b1. Installed as Add On. I have my automation integration as a package: folder/integration.yaml in which I include the main integrations.yaml from the root of my config folder. Logging says:

2022-09-01:15:28:31|ERROR[/usr/local/lib/python3.10/site-packages/hypercorn/logging.py:100@exception] Error in ASGI Framework
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/hypercorn/asyncio/task_group.py", line 21, in _handle
    await invoke_asgi(app, scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/hypercorn/utils.py", line 247, in invoke_asgi
    await app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 269, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 92, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 147, in simple_response
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/exceptions.py", line 93, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/exceptions.py", line 82, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 670, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 266, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 65, in app
    response = await func(request)
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 227, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 162, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/usr/local/lib/python3.10/site-packages/starlette/concurrency.py", line 41, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/app/src/api/routes/automations.py", line 18, in list_autos
    automations.reload()
  File "/app/src/automations/manager.py", line 45, in reload
    batch.append(next(auto_it))
  File "/app/src/automations/loader.py", line 37, in load_and_iter_automations
    for ref in extract_automation_refs(hass_config):
  File "/app/src/automations/loader.py", line 59, in extract_automation_refs
    for ref in chain(
  File "/app/src/automations/loader.py", line 100, in extract_automation_package_refs
    yield from extract_automation_inline_package_refs(
  File "/app/src/automations/loader.py", line 172, in extract_automation_inline_package_refs
    raise AssertionError(
AssertionError: configurations.homeassistant.packages[media_player] must be a dictionary!

I think your error is a bit different than this one. Mind opening a separate issue and also include there some parts of your config.yaml.

Although I think I know exactly why yours broke :o will try to make a fix in the next release.

[asosnovsky]

@jchhenderson wait I just realized. Are you running shortumation in a separate docker container not addon mode?

yes, I am running in a seperate docker container as I am running HA in docker as well. (not HA OS or supervisor).

• Check that the tag is mapped to the latest version v0.7.0

the tag in docker-compose? yes, it's 0.7.0 image: asosnovsky/shortumation-armv7:v0.7.0

• if using compose do docker compose pull && docker compose build --no-cache

made no difference, sorry.

edit: My full docker-compose (for Shortumation) is:

  shortumation: # http://dietpi:8000/web
    container_name: shortumation
    image: asosnovsky/shortumation-armv7:v0.7.0
    ports:
      - 8000:8000
    extra_hosts:
      - "host.docker.internal:host-gateway"
    volumes:
      - ./homeassistant:/config       # path is same as Homeassistant
    environment:
      - SUPERVISOR_TOKEN=<renoved>
      - HASSIO_WS=ws://host.docker.internal:8123/api/websocket

@jchhenderson do you mind posting a full log of what's going on in shortumation? i.e. docker-compose logs shortumation > logs.log and then copy/upload that logs.log to here?

[ghost]

here you go.

logs.log

[asosnovsky]

here ya go!

shortumation  | 2022-09-01:10:47:01|ERROR[/app/src/automations/loader.py:51@load_and_iter_automations] when=reading file contents,automation_path=/config/JCHH/automations/switches/switch_bedroom.yaml,error='id',automation=....

looks like the automation in /config/JCHH/automations/switches/switch_bedroom.yaml is missing an id field!

strange that HA supports this o.0

I will do some testing on my end to see how they handle it. But this is undocumented behavior on their part :/

In the meantime, if you add an id: <something> to the automation it should get picked up

[ghost]

Hi,

yes, I saw that as well. I have no IDs for any automation. I added a test ID for 1 of the automations but it made no difference.

Here is the automation...

- alias: "HA: Restart"
  id: "unique_name"
  initial_state: true #ensure automation is turned on when restarting HA
  description: ''
  trigger:
    - platform: homeassistant
      event: start
  [...]

...and here is that automation in Shortumation:

edited: Typos and clarity.

[asosnovsky]

@Sholofly so I reviewed your issue a bit more. I am pretty sure that you have an invalid configuration file.

Packages should always map to a dictionary of overrides. When using the same configuration as you I get this error in HA (which tracks with the error I am throwing in Shortumation)

also moved your issue here https://github.com/asosnovsky/Shortumation/issues/159

[asosnovsky]

@jchhenderson did you reload the automations after adding the id?

also can you confirm this behavior for the automations without an id:

[ghost]

yes, I reloaded the automations. All automations showed the same error. I am currently adding IDs too all of them anyway and will report back...

[ghost]

OK, I may have had a caching issue, as they are all showing up now. However I have a different problem. SHortumation is reporting my automations as invalid (HA is is OK with them though).

[asosnovsky]

@jchhenderson yea HA loads them but won't let you see automations that have no ids. I just finished implementing a readonly mode for automations that are missing an id. Once testing is done you should be able to use it in the next release.

[ghost]

@jchhenderson yea HA loads them but won't let you see automations that have no ids. I just finished implementing a readonly mode for automations that are missing an id. Once testing is done you should be able to use it in the next release.

Thanks, I have added IDs to all may automations now, but I am now getting the error that I posted above whereby Shortumation thinks every automation has an error. Should I create a new issue for this?

[asosnovsky]

I think you might need to reload the automations. Like in the developer tools > yaml > automations.

But in general if you can wait till the next update before opening another issue that would be better :)

[asosnovsky]

@jchhenderson for now I am closing this issue, if you are still seeing some errors please post them in another issue (or reopen this)

[ghost]

OK, thanks. I;ll wait until the next release and then open a new issue if it is still occurring.

Thanks so much!!

[ghost]

I worked out what my last error was so will raise a ticket now if I cannot find one exists 👍