The app became async-only

[AlexMKX]

What happened?

Recently noticed that all my apps have stopped working. By analysing, I found that all appdaemon functions returns coroutines, which means, the python treats them as asynchronous.

Expected result : the sync apps (functions) should be able to call functions synchonously.

Version

4.4.2

Installation type

Python virtual environment

Relevant log output

Connected to pydev debugger (build 242.18071.12)
2024-06-29 09:03:31.540927 INFO AppDaemon: AppDaemon Version 4.4.2 starting
2024-06-29 09:03:31.541140 INFO AppDaemon: Python version is 3.11.6
2024-06-29 09:03:31.541267 INFO AppDaemon: Configuration read from: ./appdaemon-dev.yaml
2024-06-29 09:03:31.541383 INFO AppDaemon: Added log: AppDaemon
2024-06-29 09:03:31.541511 INFO AppDaemon: Added log: Error
2024-06-29 09:03:31.541620 INFO AppDaemon: Added log: Access
2024-06-29 09:03:31.541726 INFO AppDaemon: Added log: Diag
2024-06-29 09:03:31.562392 INFO AppDaemon: Loading Plugin HASS using class HassPlugin from module hassplugin
2024-06-29 09:03:31.583904 INFO HASS: HASS Plugin Initializing
2024-06-29 09:03:31.584114 INFO HASS: HASS Plugin initialization complete
2024-06-29 09:03:31.584640 INFO AppDaemon: Initializing HTTP
2024-06-29 09:03:31.584990 INFO AppDaemon: Using 'ws' for event stream
2024-06-29 09:03:31.588965 INFO AppDaemon: Starting API
2024-06-29 09:03:31.592535 INFO AppDaemon: Starting Admin Interface
2024-06-29 09:03:31.592852 INFO AppDaemon: Starting Dashboards
2024-06-29 09:03:31.683163 INFO HASS: Connected to Home Assistant 2024.6.4
2024-06-29 09:03:31.742585 WARNING AppDaemon: App 'appdaemon' missing 'class' or 'module' entry - ignoring
2024-06-29 09:03:31.742887 WARNING AppDaemon: App 'http' missing 'class' or 'module' entry - ignoring
2024-06-29 09:03:31.752846 INFO AppDaemon: App 'ha_door_occupancy' added
2024-06-29 09:03:31.753712 INFO AppDaemon: Found 1 active apps
2024-06-29 09:03:31.753899 INFO AppDaemon: Found 0 inactive apps
2024-06-29 09:03:31.754075 INFO AppDaemon: Found 0 global libraries
2024-06-29 09:03:31.754263 INFO AppDaemon: Starting Apps with 1 workers and 1 pins
2024-06-29 09:03:31.755477 INFO AppDaemon: Running on port 5050
2024-06-29 09:03:31.813303 INFO HASS: Evaluating startup conditions
2024-06-29 09:03:31.817478 INFO HASS: Startup condition met: hass state=RUNNING
2024-06-29 09:03:31.817720 INFO HASS: All startup conditions met
2024-06-29 09:03:31.858089 INFO AppDaemon: Got initial state from namespace default
2024-06-29 09:03:33.764545 INFO AppDaemon: Scheduler running in realtime
2024-06-29 09:03:33.767765 INFO AppDaemon: Reading config
2024-06-29 09:03:33.788351 WARNING AppDaemon: App 'appdaemon' missing 'class' or 'module' entry - ignoring
2024-06-29 09:03:33.788863 WARNING AppDaemon: App 'http' missing 'class' or 'module' entry - ignoring
2024-06-29 09:03:33.806472 INFO AppDaemon: /home/alex/PycharmProjects/ha_door_occupancy/appdaemon-dev.yaml added or modified
2024-06-29 09:03:33.808388 INFO AppDaemon: Found 1 active apps
2024-06-29 09:03:33.808610 INFO AppDaemon: Found 0 inactive apps
2024-06-29 09:03:33.808846 INFO AppDaemon: Found 0 global libraries
2024-06-29 09:03:33.811238 INFO AppDaemon: Adding /home/alex/PycharmProjects/ha_door_occupancy to module import path
2024-06-29 09:03:33.811618 INFO AppDaemon: Adding /home/alex/PycharmProjects/ha_door_occupancy/ha_door_occupancy to module import path
2024-06-29 09:03:33.811944 INFO AppDaemon: Adding /home/alex/PycharmProjects/ha_door_occupancy/compiled to module import path
2024-06-29 09:03:33.812211 INFO AppDaemon: Adding /home/alex/PycharmProjects/ha_door_occupancy/compiled/css to module import path
2024-06-29 09:03:33.812473 INFO AppDaemon: Adding /home/alex/PycharmProjects/ha_door_occupancy/compiled/javascript to module import path
2024-06-29 09:03:33.812798 INFO AppDaemon: Adding /home/alex/PycharmProjects/ha_door_occupancy/www to module import path
2024-06-29 09:03:33.813051 INFO AppDaemon: Adding /home/alex/PycharmProjects/ha_door_occupancy/namespaces to module import path
2024-06-29 09:03:33.817141 INFO AppDaemon: Loading App Module: /home/alex/PycharmProjects/ha_door_occupancy/ha_door_occupancy/ha_door_occupancy.py
2024-06-29 09:03:33.837170 INFO AppDaemon: Loading app ha_door_occupancy using class ha_door_occupancy from module ha_door_occupancy
2024-06-29 09:03:33.839732 INFO AppDaemon: Calling initialize() for ha_door_occupancy
all_items 
PyDev console: starting.
<Task pending name='Task-229' coro=<ADAPI.get_state() running at /home/alex/PycharmProjects/ha_door_occupancy/.venv/lib/python3.11/site-packages/appdaemon/adapi.py:1454> cb=[Futures.remove_future('ha_door_occupancy')()]>

Relevant code in the app or config file that caused the issue

import datetime

import appdaemon.plugins.hass.hassapi as hass

import os
import sys
from appdaemon.__main__ import main
from pydantic_settings import BaseSettings
from pydantic import Field
from typing import Optional
import re

if __name__ == '__main__':
    cd = os.path.join(os.path.dirname(__file__), '..')
    os.chdir(cd)
    sys.argv.extend(['-c', './', '-C', 'appdaemon-dev.yaml'])
    sys.exit(main())

class ha_door_occupancy(hass.Hass):
    def initialize(self):
        all_items = self.get_state() ##<<-- this always gets a coroutine as a result

Anything else?

By stepping within the code, I found that sync_wrapper always gets event_loop

def sync_wrapper(coro) -> Callable:
    @wraps(coro)
    def inner_sync_wrapper(self, *args, **kwargs):
        is_async = None
        try:
            # do this first to get the exception
            # otherwise the coro could be started and never awaited
            asyncio.get_event_loop()
            is_async = True

The list of packages in venv

(.venv) alex@lindev:~/PycharmProjects/ha_door_occupancy$ pip list
Package            Version
------------------ ------------
aiohttp            3.8.6
aiohttp-jinja2     1.5.1
aiosignal          1.3.1
annotated-types    0.7.0
appdaemon          4.4.2
astral             3.2
async-timeout      4.0.3
attrs              23.2.0
bcrypt             4.0.1
bidict             0.23.1
certifi            2024.6.2
charset-normalizer 3.3.2
deepdiff           6.3.0
feedparser         6.0.11
frozenlist         1.4.1
h11                0.14.0
idna               3.7
iso8601            1.1.0
Jinja2             3.1.4
MarkupSafe         2.1.5
multidict          6.0.5
ordered-set        4.1.0
paho-mqtt          1.6.1
pid                3.0.4
pip                23.2.1
pydantic           2.7.4
pydantic_core      2.18.4
pydantic-settings  2.3.4
python-dateutil    2.8.2
python-dotenv      1.0.1
python-engineio    4.9.1
python-socketio    5.8.0
pytz               2023.3.post1
PyYAML             6.0.1
requests           2.28.2
setuptools         68.2.0
sgmllib3k          1.0.0
simple-websocket   1.0.0
six                1.16.0
sockjs             0.11.0
tomli              2.0.1
tomli_w            1.0.0
typing_extensions  4.12.2
urllib3            1.26.19
uvloop             0.17.0
websocket-client   1.5.3
wheel              0.41.2
wsproto            1.2.0
yarl               1.9.4

[acockburn]

Hi there - nothing in AD has changed in this area for several years, and AD itself hasn't had a release in over a year, so if your apps recently stopped working, the cause must be elsewhere.

AD is designed so that all callbacks are tested to see if they are async, and if so, they are called asynchronously by AD using the main event loop, if not they are called synchronously in the Apps thread.

AD api calls are all synchronous to the calling app and run in the app's thread, but call into AD's core which is asynchronous under the hood.

This code has been tested and used by thousands of people for several years so I am reasonably confident there isn't a problem here ;)

You don't give any details go how your apps broke, or any error messages. In your example app, logging the output of the get_state() command would prove that initialize() is running correctly, and that the get_state() was working as expected for instance. Until we have a better idea of what exactly is going wrong it's hard to suggest a way to fix it.

[AlexMKX]

Thank you for the clarifications. It just "have been stopped" when I've rebuilt the container. Perhaps it is because of the new python version or so. I'll try to dig deeper within docker and specific versions to find a root cause. At this moment I just unable to run anything synchronously, so I've been forced to rewrite all my stuff to async. Seems it async is more trendy :)

[acockburn]

Definitely trendy but not necessarily better :) AD works either way, but I generally would only use async if a specific library needs it for instance, for everything else the synchronous threaded model works great.

Start simple with some logging like I suggested and build up from there. Good luck!

[AlexMKX]

The first one is caught. PyCharm introduced the asyncio debug feature. In this case it seems event loop is created when doing debug. https://youtrack.jetbrains.com/issue/PY-57667/Asyncio-support-for-the-debugger-EXPERIMENTAL-FEATURE By disabling this feature, the appdaemon apps goes synchronous mode. However, as hass addon it still async. Doing more research.