RuntimeError: asyncio.run() cannot be called from a running event loop

[mr-lanholmes]

Hi! Was running the code below and got into this error.

KASA_USERNAME = '###'
KASA_PASSWORD = '###'
device_manager = TPLinkDeviceManager(KASA_USERNAME, KASA_PASSWORD)
devices = device_manager.get_devices()

Error shown:

Traceback (most recent call last):
    device_manager = TPLinkDeviceManager(KASA_USERNAME, KASA_PASSWORD)
  File "#\lib\site-packages\tplinkcloud\device_manager.py", line 41, in __init__
    self.get_devices()
  File "#\lib\site-packages\tplinkcloud\device_manager.py", line 97, in get_devices
    return asyncio.run(self._fetch_devices())
  File "#\lib\asyncio\runners.py", line 33, in run
    raise RuntimeError(
RuntimeError: asyncio.run() cannot be called from a running event loop

Tried clearing the asyncio queue but still ran to this problem.

Could anyone please help with this?

Thanks!

[mr-lanholmes]

Device: KP105 https://www.tp-link.com/uk/home-networking/smart-plug/kp105/

[piekstra]

@mr-lanholmes are you perhaps running this in the context of a jupyter notebook?

If so, you maybe running into the issue described here: https://stackoverflow.com/questions/55409641/asyncio-run-cannot-be-called-from-a-running-event-loop

To get around this, you could construct the manager with prefetch=False like so:

device_manager = TPLinkDeviceManager(KASA_USERNAME, KASA_PASSWORD, prefetch=False)

Then, when trying to get devices, you could try leveraging the internal _fetch_devices method directly instead of using get_devices. With that, you would simply do

devices = await device_manager._fetch_devices()

Let me know if that works. That's a bit of a hack, so I may need to setup a new version for the manager that has specifically different async vs synchronous methods.

It looks like there should also be a way to check if there is an asyncio event loop already running before using the device manager. As mentioned in the Stack Overflow post:

import asyncio

try:
    loop = asyncio.get_running_loop()
except RuntimeError:  # if cleanup: 'RuntimeError: There is no current event loop..'
    loop = None

if loop and loop.is_running():
    print('Async event loop already running')

[mr-lanholmes]

@piekstra Thanks for your kind reply!

1. Yea, I am running the code from Spyder IDE (Jupyter Console)
2. The prefetch=False helped to solve the async problem I have, however when trying to access the device by simply devices[0] I'm running to TypeError: 'coroutine' object is not subscriptable issue. It seems the ._fetch_devices() method return an async item rather than the device array?

Thanks!

Have a nice day!

[piekstra]

@mr-lanholmes Thanks for confirming your environment! That helps a lot.

As for the coroutine issue - that is definitely odd. There may be more subtleties to asyncio than I am currently aware of which are causing the problems. I will try and spend some time tomorrow working out an alternative version of the device manager, or add new methods to the existing one, that allows for you to perform operations synchronously and leverage your own async operations.

[piekstra]

As an update, I've found some posts on Stack Overflow indicating there may be jupyter-specific workarounds and am planning to set up my environment to test for those options. I'd rather avoid doing anything too specific to jupyter and instead find a solution that is backwards compatible for existing users of the library but also allows easy usage from within the context of a jupyter notebook.

[piekstra]

@mr-lanholmes I've just now had some time and set up a Jupyter notebook environment to begin testing changes. I hope to have a fix figured out today. Sorry for the delays.

[piekstra]

@mr-lanholmes it's sort of a workaround, but I want to maintain the library as-is and will add to the documentation a note on the following, but:

Jupyter Notebooks running Python 3 do not allow asyncio.run to be called, because the notebook already has an asyncio event loop running and (docs):

This function cannot be called when another asyncio event loop is running in the same thread.

To get around this, the easiest thing to do is to create a new thread for any methods that you need to run where asyncio.run is called. For example:

import threading
from tplinkcloud import TPLinkDeviceManager

username = 'REDACTED'
password = 'REDACTED'

device_manager = TPLinkDeviceManager(username, password, verbose=True, prefetch=False)

devices_thread = threading.Thread(target=device_manager.get_devices)
devices_thread.start()
devices = devices_thread.join()

I verified this to work in a Jupyter Notebook context

[piekstra]

The docs now discuss the workaround here: https://github.com/piekstra/tplink-cloud-api#jupyter-notebooks