async / await methods?

[ClusterM]

Python 3.5 introduced async methods. It's very useful for network programming in multitask applications (like Home Assistant). But TinyTuya doesn't use it. Are there any plans to add async methods?

[jasonacox]

HI @ClusterM , I'm open to suggestions. TinyTuya was originally targeted at Python 2.7 and 3.x, but I'm open to a shift provided the approach stays accessible to the masses. I have seen a lot of async code that is so obscure and obtuse that you need a PhD in the module to use the module. But to be fair, you could do that with any project. ;-)

[jasonacox]

One approach we could consider... create an optional module for async functionality that would be HA compatible. Thoughts?

[make-all]

Probably you'd want to do it the other way around - convert to use asyncio and provide an optional module with synchronous wrappers that call the async functions and await the result. Making the async ones the wrappers would mean you are using synchronous network io underneath, but simulating async using threads, and could get quite messy to code and deal with concurrency etc.

[uzlonewolf]

@make-all The issue is we still support Python 2.7 and early 3.x which makes wrapping it that way kinda difficult. The need to async the entire chain of function calls all the way down doesn't help things either. IMO they really screwed up by requiring that, there's no good way of supporting both asyncio and non-asyncio because of it.

[make-all]

Its ultimately your own decision whether to keep supporting old EOL versions of Python for some hypothetical users who want to run new versions of libraries on old versions of Python, but personally I would be drawing the line at 3.7 in 2023.

It took me a while to wrap my head around Python's implementation of async, and the cascading use of await/async bothered me at first, but understanding that it is effectively cooperative multitasking and those keywords are telling you where the context can switch helps to understand why they designed it the way they did.

[uzlonewolf]

I understand what it's doing, my issue is with how it was implemented. Currently there is no way to use the same code in both async-mode and non-async-mode, you can only do one and must completely rewrite it to switch to the other. Had they used @​decorators and made it await(...) you could easily implement those as null functions if you didn't want to use async.

Older versions of Python aside, completely rewriting the library and telling every user they also must completely rewrite their code if they want to keep using it is not a trivial change. If you know how to make it so you can do

d = Device(...)
print( d.status() )

in both current non-async programs as well as new async ones without duplicating every single function in every single device type then I'm all ears.

[jasonacox]

Thanks @make-all. Fair points, especially about Python 2.7. This is something we need to consider.

Python async on the other hand, is the bigger issue, as @uzlonewolf mentions. Besides API consistency issues, async is a bit intimidating and requires a steep learning curve. Even the terminology can be intimidating. Just to be clear, I'm not disparaging anyone who learns and loves the technology. The fact that it's difficult shouldn't limit anyone from learning it and I encourage everyone to do so. However, a majority in this community (by the issues raised and interactions I have had) are brilliant IoT makers, hackers, and enthusiast that are often python novice.

From the beginning, I wanted to make TinyTuya approachable (where "Tiny" is a way of saying simple, small, elegant and widely manageable). I wanted it to be easy to understand, learn and use. I want even novice python users to be able to hand-craft their own automations and even contribute to the project itself. Most of all, I wanted this to be FUN! I see so many projects that start great but become a burden for the maintainers and community, with grueling, complex issues, and sadly after a while, no fun. I don't want that for us. I'm not saying async would do that, but it would present challenges that could lead us down that path.

I really do appreciate this feedback. Feedback is gold! We do want to be challenged even if it is about our mission or how to get there. And we may be convinced to change course. It may make sense to deliver elegance through a more complex, clever and performant way like async. We may want to hand a different, more robust API to more advanced and expert python users. But I'm also good with keeping TinyTuya small, simple, easy to understand... and of course, fun. ;)

Things to consider... 🙏