Is this library sync or async? Maybe a mix?

[lyricnz]

Q: How to use this library in an asyncio way?

While investigating https://github.com/lyricnz/tplink_ess/issues/22 I saw that the integration is taking a lot of time in the event loop of HA. I think this library is entirely synchronous io, because it uses:

self.r_socket.recvfrom(1500)
self.s_socket.sendto(packet, (Network.BROADCAST_ADDR, Network.UDP_SEND_TO_PORT))

[firstof9]

it's not blocking, the timeout can be reduced

https://github.com/firstof9/tplink-ess-lib/blob/3d47471687d0af5ec73db8c566c3b66d9e8a8ed4/tplink_ess_lib/network.py#L63

here, but this gets it's own thread when queries happen AFAIK

[lyricnz]

If this was really async, then wouldn't these tasks run in parallel?

    tasks = []
    for switch in switches:
        logging.info("Querying %s", switch["hostname"])
        tasks.append(query_switch(api, switch, "stats"))
        tasks.append(query_switch(api, switch, "ports"))
        tasks.append(query_switch(api, switch, "vlan"))
    results = await asyncio.gather(*tasks)

But it looks like they're running in strict sequence (in the main thread):

MainProcess MainThread 2023-04-28 08:16:09,286 INFO     Querying DONE TL-SG108E for stats
MainProcess MainThread 2023-04-28 08:16:09,286 INFO     Querying TL-SG108E for ports
MainProcess MainThread 2023-04-28 08:16:09,402 INFO     Querying DONE TL-SG108E for ports
MainProcess MainThread 2023-04-28 08:16:09,402 INFO     Querying TL-SG108E for vlan
MainProcess MainThread 2023-04-28 08:16:09,497 INFO     Querying DONE TL-SG108E for vlan
MainProcess MainThread 2023-04-28 08:16:09,497 INFO     Querying Simon-TL-SG105PE for stats
MainProcess MainThread 2023-04-28 08:16:09,669 INFO     Querying DONE Simon-TL-SG105PE for stats
MainProcess MainThread 2023-04-28 08:16:09,669 INFO     Querying Simon-TL-SG105PE for ports
MainProcess MainThread 2023-04-28 08:16:09,776 INFO     Querying DONE Simon-TL-SG105PE for ports
MainProcess MainThread 2023-04-28 08:16:09,776 INFO     Querying Simon-TL-SG105PE for vlan
MainProcess MainThread 2023-04-28 08:16:09,866 INFO     Querying DONE Simon-TL-SG105PE for vlan
MainProcess MainThread 2023-04-28 08:16:09,866 INFO     Querying Back-TL-SG105PE for stats
MainProcess MainThread 2023-04-28 08:16:10,040 INFO     Querying DONE Back-TL-SG105PE for stats
MainProcess MainThread 2023-04-28 08:16:10,040 INFO     Querying Back-TL-SG105PE for ports
MainProcess MainThread 2023-04-28 08:16:10,144 INFO     Querying DONE Back-TL-SG105PE for ports
MainProcess MainThread 2023-04-28 08:16:10,145 INFO     Querying Back-TL-SG105PE for vlan```

Is this caused by the locking/mutex 

[firstof9]

If you want we can give that a shot, I'm also looking to add a parameter to the call as a list to limit the query to whatever's configured as well.

[lyricnz]

Yeah, limiting the set of queries would be helpful (right now, the integation does "all of them" but only uses a couple). That should be a fairly simple patch.

I'm fairly new to asyncio - need to understand the correct way to do this for datagrams. We don't want to be blocking the main thread for even 1/10th of a second, let alone 10s.

[lyricnz]

Further evidence that the code is not running async: Adding a task to top of tasks[] in the code above that does:

async def hello():
    logging.info('Hello ...')
    await asyncio.sleep(0.1)
    logging.info('... World!')

Ends up printing:

MainProcess MainThread 2023-04-28 13:34:22,534 INFO     Hello ...
MainProcess MainThread 2023-04-28 13:34:22,535 INFO     Querying TL-SG108E for stats
MainProcess MainThread 2023-04-28 13:34:22,746 INFO     Querying DONE TL-SG108E for stats
MainProcess MainThread 2023-04-28 13:34:22,746 INFO     Querying TL-SG108E for ports
...
MainProcess MainThread 2023-04-28 13:34:23,686 INFO     Querying DONE Simon-TL-SG105PE for vlan
MainProcess MainThread 2023-04-28 13:34:23,686 INFO     ... World!

[firstof9]

We don't want to be blocking the main thread for even 1/10th of a second, let alone 10s.

If we were doing that HA would let us know, HA continues on while we poll the switches.

[lyricnz]

I'm not convinced about that (HA letting us know; or doing things in a different thread), given then output of the profiler at https://github.com/lyricnz/tplink_ess/issues/22#issue-1684225260

And when I add the thread-name to the log message it emits MainThread when doing the tx/rx.