Display Python asyncio Tasks in VS Code Debugger

[mschurr]

Please feel free to redirect this feature request to the appropriate spot, as I had difficulty determining where that is.

I would like to have better support for debugging asyncio event loops. Specifically, I would like asyncio.Tasks to appear in the CALL STACK section of the Python debugger, alongside the existing support for processes and threads.

Here's an example that shows how it's possible to retrieve information about running tasks from an asyncio EventLoop that would be fantastic to display in the debugger:

import asyncio
import os
import signal
import sys

async def example_task() -> None:
    while True:
        await asyncio.sleep(10)

async def main() -> None:
    loop = asyncio.get_running_loop()
    add_interrupt_handler(loop)
    add_sigusr1_handler(loop)

    tasks = [asyncio.create_task(example_task(), name=f"sleeper_{i}") for i in range(10)]

    await asyncio.sleep(1)
    os.kill(os.getpid(), signal.SIGUSR1)

    await asyncio.gather(*tasks, return_exceptions=True)

def dump_tasks(loop: asyncio.AbstractEventLoop) -> None:
    for task in asyncio.all_tasks(loop=loop):
        print(f"----- Task {task.get_name()} -----", file=sys.stderr, flush=True)
        # see also: https://docs.python.org/3/library/asyncio-task.html#asyncio.Task.get_stack
        task.print_stack(limit=50, file=sys.stderr)
        sys.stderr.flush()

def add_sigusr1_handler(loop: asyncio.AbstractEventLoop) -> None:
    loop.add_signal_handler(signal.SIGUSR1, dump_tasks, loop)

def add_interrupt_handler(loop: asyncio.AbstractEventLoop) -> None:
    task = asyncio.current_task()
    assert task is not None
    task.set_name("main")
    loop.add_signal_handler(signal.SIGTERM, task.cancel, "SIGTERM")
    loop.add_signal_handler(signal.SIGINT, task.cancel, "KEYBOARD_INTERRUPT")

if __name__ == "__main__":
    asyncio.run(main(), debug=True)

This outputs:

----- Task sleeper_1 -----
Stack for <Task pending name='sleeper_1' coro=<example_task() running at /home/mschurr/example.py:9> wait_for=<Future pending cb=[Task.task_wakeup()] created at /usr/lib/python3.10/asyncio/base_events.py:429> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720] created at /usr/lib/python3.10/asyncio/tasks.py:337> (most recent call last):
  File "/home/mschurr/example.py", line 9, in example_task
    await asyncio.sleep(10)
----- Task sleeper_4 -----
Stack for <Task pending name='sleeper_4' coro=<example_task() running at /home/mschurr/example.py:9> wait_for=<Future pending cb=[Task.task_wakeup()] created at /usr/lib/python3.10/asyncio/base_events.py:429> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720] created at /usr/lib/python3.10/asyncio/tasks.py:337> (most recent call last):
  File "/home/mschurr/example.py", line 9, in example_task
    await asyncio.sleep(10)
----- Task sleeper_9 -----
Stack for <Task pending name='sleeper_9' coro=<example_task() running at /home/mschurr/example.py:9> wait_for=<Future pending cb=[Task.task_wakeup()] created at /usr/lib/python3.10/asyncio/base_events.py:429> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720] created at /usr/lib/python3.10/asyncio/tasks.py:337> (most recent call last):
  File "/home/mschurr/example.py", line 9, in example_task
    await asyncio.sleep(10)
----- Task sleeper_0 -----
Stack for <Task pending name='sleeper_0' coro=<example_task() running at /home/mschurr/example.py:9> wait_for=<Future pending cb=[Task.task_wakeup()] created at /usr/lib/python3.10/asyncio/base_events.py:429> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720] created at /usr/lib/python3.10/asyncio/tasks.py:337> (most recent call last):
  File "/home/mschurr/example.py", line 9, in example_task
    await asyncio.sleep(10)
----- Task sleeper_7 -----
Stack for <Task pending name='sleeper_7' coro=<example_task() running at /home/mschurr/example.py:9> wait_for=<Future pending cb=[Task.task_wakeup()] created at /usr/lib/python3.10/asyncio/base_events.py:429> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720] created at /usr/lib/python3.10/asyncio/tasks.py:337> (most recent call last):
  File "/home/mschurr/example.py", line 9, in example_task
    await asyncio.sleep(10)
----- Task main -----
Stack for <Task pending name='main' coro=<main() running at /home/mschurr/example.py:22> wait_for=<_GatheringFuture pending cb=[Task.task_wakeup()] created at /usr/lib/python3.10/asyncio/tasks.py:665> cb=[_run_until_complete_cb() at /usr/lib/python3.10/asyncio/base_events.py:184] created at /usr/lib/python3.10/asyncio/tasks.py:636> (most recent call last):
  File "/home/mschurr/example.py", line 22, in main
    await asyncio.gather(*tasks, return_exceptions=True)
----- Task sleeper_2 -----
Stack for <Task pending name='sleeper_2' coro=<example_task() running at /home/mschurr/example.py:9> wait_for=<Future pending cb=[Task.task_wakeup()] created at /usr/lib/python3.10/asyncio/base_events.py:429> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720] created at /usr/lib/python3.10/asyncio/tasks.py:337> (most recent call last):
  File "/home/mschurr/example.py", line 9, in example_task
    await asyncio.sleep(10)
----- Task sleeper_5 -----
Stack for <Task pending name='sleeper_5' coro=<example_task() running at /home/mschurr/example.py:9> wait_for=<Future pending cb=[Task.task_wakeup()] created at /usr/lib/python3.10/asyncio/base_events.py:429> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720] created at /usr/lib/python3.10/asyncio/tasks.py:337> (most recent call last):
  File "/home/mschurr/example.py", line 9, in example_task
    await asyncio.sleep(10)
----- Task sleeper_3 -----
Stack for <Task pending name='sleeper_3' coro=<example_task() running at /home/mschurr/example.py:9> wait_for=<Future pending cb=[Task.task_wakeup()] created at /usr/lib/python3.10/asyncio/base_events.py:429> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720] created at /usr/lib/python3.10/asyncio/tasks.py:337> (most recent call last):
  File "/home/mschurr/example.py", line 9, in example_task
    await asyncio.sleep(10)
----- Task sleeper_8 -----
Stack for <Task pending name='sleeper_8' coro=<example_task() running at /home/mschurr/example.py:9> wait_for=<Future pending cb=[Task.task_wakeup()] created at /usr/lib/python3.10/asyncio/base_events.py:429> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720] created at /usr/lib/python3.10/asyncio/tasks.py:337> (most recent call last):
  File "/home/mschurr/example.py", line 9, in example_task
    await asyncio.sleep(10)
----- Task sleeper_6 -----
Stack for <Task pending name='sleeper_6' coro=<example_task() running at /home/mschurr/example.py:9> wait_for=<Future pending cb=[Task.task_wakeup()] created at /usr/lib/python3.10/asyncio/base_events.py:429> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720] created at /usr/lib/python3.10/asyncio/tasks.py:337> (most recent call last):
  File "/home/mschurr/example.py", line 9, in example_task
    await asyncio.sleep(10)

^CTraceback (most recent call last):
  File "/usr/lib/python3.10/asyncio/tasks.py", line 605, in sleep
    return await future
asyncio.exceptions.CancelledError: KEYBOARD_INTERRUPT

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mschurr/example.py", line 22, in main
    await asyncio.gather(*tasks, return_exceptions=True)
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mschurr/example.py", line 45, in <module>
    asyncio.run(main(), debug=True)
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
asyncio.exceptions.CancelledError

[int19h]

Do you expect them to show up as logical threads directly under process nodes?

[mschurr]

I think it's more valuable that they show up at all than where specifically they show up.

That said, it probably makes the most sense to nest them under the OS thread running the event loop if possible?

[int19h]

Having them show up as children under the event loop thread (or threads, since there can be multiple - I totally forgot about that!) makes a lot of sense.

Problem is, on the debugpy end of things, we can only do as much as DAP permits - and it doesn't have the notion of async threads as something distinct, nor does it allow to nest threads. So, we need to update the DAP protocol to have some way to represent all this in a language-agnostic way, and then VSCode needs to support those additions.

[VincentVanlaer]

Something to take into account as well is structured concurrency, where tasks are nested in a tree. This is the case for trio as well as the new TaskGroup api. It would be nice if future changes to the DAP spec would include support for these kind of relations between async tasks, so that the client can visualize them as a tree.