Introduction of asyncio causes RuntimeError exception in multi-threaded application

[hperrey]

• Python State Machine version: 2.3.0
• Python version: 3.11.8
• Operating System: Linux/Ubuntu 6.5.0-35-generic

Description

In a multi-threaded application, I am receiving and parsing commands on one thread which can trigger state machine changes. These are monitored on other threads by checking the state of the FSM. This design worked fine on versions before the recent introduction of asyncio but now result in RuntimeErrors related to the event loop.

What I Did

As a very simple reproducer, the following code demonstrates the issue I am facing:

import threading
from statemachine import StateMachine, State
from time import sleep

class TrafficLightMachine(StateMachine):
    "A traffic light machine"
    green = State(initial=True)
    yellow = State()
    red = State()

    cycle = (
        green.to(yellow)
        | yellow.to(red)
        | red.to(green)
    )

class Controller:

    def __init__(self):
        self.fsm = TrafficLightMachine()
        # set up thread
        t = threading.Thread(target=self.recv_cmds)
        t.start()

    def recv_cmds(self):
        """Pretend we receive a command triggering a state change after 3s."""
        sleep(3)
        self.fsm.cycle()

    def print_status(self):
        wait = 4
        while wait > 0:
            print(self.fsm.current_state.id)
            sleep(0.5)
            wait -= 0.5

if __name__ == '__main__':
    c = Controller()
    c.print_status()

Running the above code with python-statemachine 2.3.0 results in the following output:

$ python test_fsm.py 
green
green
green
green
green
green
Exception in thread Thread-1 (recv_cmds):
Traceback (most recent call last):
  File "/home/hperrey/mambareforge/envs/cstl_update/lib/python3.11/site-packages/statemachine/utils.py", line 29, in run_async_from_sync
    loop = asyncio.get_running_loop()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: no running event loop

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/hperrey/mambareforge/envs/cstl_update/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
    self.run()
  File "/home/hperrey/mambareforge/envs/cstl_update/lib/python3.11/threading.py", line 982, in run
    self._target(*self._args, **self._kwargs)
  File "/home/hperrey/src/constellation/python/tests/test_fsm.py", line 31, in recv_cmds
    self.fsm.cycle()
  File "/home/hperrey/mambareforge/envs/cstl_update/lib/python3.11/site-packages/statemachine/event.py", line 56, in trigger_event
    return run_async_from_sync(event_instance.trigger(self, *args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hperrey/mambareforge/envs/cstl_update/lib/python3.11/site-packages/statemachine/utils.py", line 32, in run_async_from_sync
    loop = asyncio.get_event_loop()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hperrey/mambareforge/envs/cstl_update/lib/python3.11/asyncio/events.py", line 681, in get_event_loop
    raise RuntimeError('There is no current event loop in thread %r.'
RuntimeError: There is no current event loop in thread 'Thread-1 (recv_cmds)'.
green
/home/hperrey/mambareforge/envs/cstl_update/lib/python3.11/threading.py:1047: RuntimeWarning: coroutine 'Event.trigger' was never awaited
  self._invoke_excepthook(self)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
green

while version 2.1.2 resulted in the expected

green
green
green
green
green
green
yellow
yellow

I would be very grateful for any advice you might have on how to fix this. I am not really familiar with asyncio and there might be an easy way around this issue that I just don't see.

While the code above is of course far from perfect, the general design (FSM progressed from another thread) is something I am stuck with at the moment in more than one project -- the more I hope you might have an idea of how to make the code work even with more recent versions of python-statemachine!

[fgmacedo]

Hi @hperrey,

Thank you for reporting this issue. It appears to be a regression, as I did not bump the major version according to SemVer, and this release should not have introduced any breaking changes.

I've identified that an automated test for "official support for multi-threaded applications" was missing, which is related to an existing issue: https://github.com/fgmacedo/python-statemachine/issues/403.

I appreciate the code snippet you provided. I will use it to reproduce and address the error.

Here are the next steps:

1. If I can reproduce and fix the error, we will release a patch/hotfix.
2. If the issue cannot be resolved quickly, I will publish a patch release that reverts the 2.3.0 changes, allowing more time to find a fix.

Thank you again!

[fgmacedo]

Hi @hperrey , I've just released a patch version 2.3.1 that fixes this issue: https://pypi.org/project/python-statemachine/2.3.1/

Can you check if it works for you?

Thanks again for your detailed report, I was able to quickly reproduce the issue.

Best!

[hperrey]

@fgmacedo this does indeed fix the issues I observed in all projects! Thank you so much for looking into this and even providing a fix on such a short timescale! Much obliged :)