Crashes on 0.71.0 when pushing/popping screens quickly

[darrenburns]

Since updating to 0.71.0, I'm getting crashes when I quickly push and pop screens.

I've updated all of my code to use await the newly returned awaitables.

This is the exception that I see:

Traceback (most recent call last):
  File "/Users/darrenburns/Code/posting/.venv/bin/posting", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/Users/darrenburns/Code/posting/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/darrenburns/Code/posting/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/darrenburns/Code/posting/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/darrenburns/Code/posting/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/darrenburns/Code/posting/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/darrenburns/Code/posting/src/posting/__main__.py", line 64, in default
    app.run()
  File "/Users/darrenburns/Code/posting/.venv/lib/python3.11/site-packages/textual/app.py", line 1624, in run
    asyncio.run(run_app())
  File "/Users/darrenburns/.rye/py/cpython@3.11.7/install/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/Users/darrenburns/.rye/py/cpython@3.11.7/install/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/darrenburns/.rye/py/cpython@3.11.7/install/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/darrenburns/Code/posting/.venv/lib/python3.11/site-packages/textual/app.py", line 1610, in run_app
    await self.run_async(
  File "/Users/darrenburns/Code/posting/.venv/lib/python3.11/site-packages/textual/app.py", line 1574, in run_async
    await asyncio.shield(app._shutdown())
  File "/Users/darrenburns/Code/posting/.venv/lib/python3.11/site-packages/textual/app.py", line 2831, in _shutdown
    await self._close_all()
  File "/Users/darrenburns/Code/posting/.venv/lib/python3.11/site-packages/textual/app.py", line 2809, in _close_all
    await self._prune_node(stack_screen)
  File "/Users/darrenburns/Code/posting/.venv/lib/python3.11/site-packages/textual/app.py", line 3479, in _prune_node
    raise asyncio.TimeoutError(
TimeoutError: Timeout waiting for [ToastRack(id='textual-toastrack'), Tooltip(id='textual-tooltip'), AppHeader(), UrlBar(), AppBody(), Footer(), AutoComplete(), AutoComplete()] to close; possible deadlock (consider changing App.CLOSE_TIMEOUT)

I've also noticed a couple of situations where this "possible deadlock" error is completely hiding a real underlying error (I think it was when there was an exception in my compose method). I had to manually catch the exception to see what it was.

[willmcgugan]

I'm not sure the deadlock detection is hiding an error. Without the timeout, it would have frozen.

If a broken compose prevents a mount or prune, we might have to handle that case explicitly.

[darrenburns]

It was definitely hiding an error (to be clear, I mean an error in my app, not Textual, inside a widget's compose method. By "hiding", I mean there was no stack trace printed by Textual anywhere indicating why my app crashed, just the TimeoutError) - I wrapped the code that contained an exception in a try/except and printed out the exception.

When I didn't catch the error myself, it bubbled up to Textual and the app crashed with the deadlock issue after 5 seconds, and there was no sign of the original error.

[darrenburns]

I will try to MRE this tomorrow.

[darrenburns]

Have spent some time trying and failing to MRE this, but I'm pretty certain there's an issue.

My app had a logic bug which caused a plain old exception to be raised, but Textual never presented that exception to the user. The user was only presented with "possible deadlock" when the app crashed.

[darrenburns]

I just hit this issue again - receiving a "possible deadlock" which is hiding another exception in my app, making debugging difficult.

I think it's unrelated to screens and is a more general issue - trying another MRE now.

[arcivanov]

Override _handle_exception in the App:

E.g.:

    def _handle_exception(self, error: Exception) -> None:
        global exc
        exc = error
        super()._handle_exception(error)

It'll allow you to see the errors

[darrenburns]

I've also noticed that I get this warning in the devtools each time I see the deadlock issue:

[14:45:01] WARNING                                            _callback.py:73
Callback functools.partial(<function
Footer.on_mount.<locals>.bindings_changed at 0x1047f87c0>, MainScreen()) is
still pending after 3 seconds

[darrenburns]

Seems to not be restricted to 0.71.0.

@arcivanov The issue here is exceptions are not making it into Textual's usual error handling flow, so this doesn't do anything in this case.

[JoeanAmier]

I also encountered this error, and when the same operation was performed, this error can be 100% reproduced.

Traceback (most recent call last):
  File "C:\Users\youyq\PycharmProjects\XHS-Downloader\main.py", line 75, in <module>
    run(app())
  File "C:\Program Files\Python312\Lib\asyncio\runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python312\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python312\Lib\asyncio\base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "C:\Users\youyq\PycharmProjects\XHS-Downloader\main.py", line 61, in app
    await xhs.run_async()
  File "C:\Users\youyq\PycharmProjects\XHS-Downloader\venv\Lib\site-packages\textual\app.py", line 1572, in run_async
    await app._shutdown()
  File "C:\Users\youyq\PycharmProjects\XHS-Downloader\venv\Lib\site-packages\textual\app.py", line 2804, in _shutdown
    await self._close_all()
  File "C:\Users\youyq\PycharmProjects\XHS-Downloader\venv\Lib\site-packages\textual\app.py", line 2784, in _close_all
    await self._prune_node(stack_screen)
  File "C:\Users\youyq\PycharmProjects\XHS-Downloader\venv\Lib\site-packages\textual\app.py", line 3445, in _prune_node
    raise asyncio.TimeoutError(
TimeoutError: Timeout waiting for [ToastRack(id='textual-toastrack'), Tooltip(id='textual-tooltip'), Header(), Label(), Label(), Label(), Label(), Label(), Label(), Label(), Label(), Footer()] to close; possible deadlock (consider changing App.CLOSE_TIMEOUT)

[JoeanAmier]

I downgraded it to 0.63.0 and found that abnormalities only occur when switching screens quickly. If I pause on the screen for a few seconds before switching to another screen, there will be no abnormalities.

[darrenburns]

I'm going to close this for now - I think it's resolved by 0.72.0. If anyone here thinks otherwise let us know!

[github-actions[bot]]

Don't forget to star the repository!

Follow @textualizeio for Textual updates.