Ctrl+C behavior problems at the new REPL

richardsheridan commented 1 month ago

It seemed a little too good to be true that Ctrl+C "just works" in the new Trio REPL (#2972, #3002), it turns out there are a couple weird problems. Interrupting running sync and async code seems to go fine, but when you are just sitting at the input prompt, KI cannot reach that code in the "usual" fashion. I've noticed 3 issues so far:

`Runner._ki_pending` flag survives returning to the prompt

Consider executing a simple uninterruptible operation:

>>> await trio.to_thread.run_sync(time.sleep, 5)

This should and does last 5 seconds even if you try to interrupt it. However, Trio is still trying to get rid of the KI hot potato, so the next checkpoint will fire off a KI:

>>> await trio.sleep(0)
 Traceback (most recent call last):
  File "<console>", line 1, in <module>
  <snip>
    raise KeyboardInterrupt
KeyboardInterrupt
>>>

This one seems to be easy enough to solve, either dig into the Runner and manually reset the flag every time we're done processing input, or run trio.from_thread.run(trio.lowlevel.checkpoint_if_cancelled) and it will safely propagate to the normal console KI code. However, after doing that exposed the next two issues.

`input` on Windows sees Ctrl+C even though there is a handler AND it's not in the main thread

If you hit Ctrl+C any time the console is waiting for input, the Trio run ends:

>>> {Ctrl+C}
now exiting TrioInteractiveConsole...
Traceback (most recent call last):
  <snip>
    raise runner.main_task_outcome.error
KeyboardInterrupt
^C
C:\>

(Weirdly, this doesn't happen during PyCharm's terminal emulation!) This is very simply a result of input popping out EOFError as if you'd done Ctrl+Z+Enter:

C:\>python -c "import trio; trio.run(trio.to_thread.run_sync, input)" {then hit Ctrl+C}
Traceback (most recent call last):
  <snip>
    ret = context.run(sync_fn, *args)
EOFError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  <snip>
    raise runner.main_task_outcome.error
KeyboardInterrupt

C:\>

I suppose in the main thread this is overridden by the KeyboardInterrupt at some point for the python REPL behavior to work right. This seems like a matter of transforming the Exception in our console subclass:

    def raw_input(self, prompt=""):
        try:
            return input(prompt)
        except EOFError:
            # check if trio has a pending KI
            trio.from_thread.run(trio.lowlevel.checkpoint_if_cancelled)
            raise

This works fine as far as I can tell.

`input` on Linux DOES NOT see Ctrl+C (because it's not in the main thread)

This isn't so bad as it only means KI at the input prompt doesn't take effect until you finish entering a (possibly multiline) command, and even then the command runs to completion. This is weird, but maybe tolerable? I think the worst part is that you can't quickly discard/drop out of multiline edits. Furthermore, I have no idea how to fix this.

Summary

This is an issue instead of a PR because there might be a smarter way to interact with KI that would also work on linux, and I wanted other people to follow up on this after experimenting a bit if there's any other weird behavior that isn't covered by the KI above checks (such as the linux case).

A5rocks commented 1 month ago

Obvious disclaimer that I don't really know/remember trio internals + how KI is delivered.

I assume there's issues running the trio loop off the main thread? (we could probably take a trio token from it and then use the same strategy as we do now, just whether a thread is main or not is reversed)

richardsheridan commented 4 weeks ago

Yes, running the trio loop in a non-main thread will break the ability to interrupt "normal" running code on all platforms, at least without some additional work.

oremanj commented 4 weeks ago

The additional work is not too bad: wrap each thing you submit to the Trio loop in a cancel scope, and on SIGINT, cancel it.

richardsheridan commented 4 weeks ago

Will that interrupt while 1: pass?

Edit: sorry, that sounds a bit snooty as I read it again. The system is complex enough that I am genuinely unsure if the trio signal handler will get called or not in that situation.

oremanj commented 4 weeks ago

Good point, it won't. The only way to interrupt that on a non-main thread is https://docs.python.org/3/c-api/init.html#c.PyThreadState_SetAsyncExc . It is not actually necessary to write one's own C extension; you can call it via ctypes. However, there are some major caveats that probably make this not workable:

it doesn't work on PyPy
it just immediately raises the exception, like KeyboardInterrupt without Trio's KI protection logic, and there's no way to replicate Trio's KI protection logic safely from a different thread; you can inspect the stack, but because the exception delivery is asynchronous, the exception might not be raised at the point you decided was safe

Solving the problems that appear when taking input on the non-main thread is probably easier than solving the problems that appear when running Trio code on the non-main thread.

A5rocks commented 2 weeks ago

So the new asyncio REPL (or rather, porting python -m asyncio to the new REPL, which is probably something we should copy because multiline editing!!) seems to detect \x03 and then pretends that is a keyboard interrupt. Maybe this is a more... robust solution, even if it doesn't feel very principled? Does it even work on Windows?

richardsheridan commented 1 week ago

Note to self, fcntl.ioctl(sys.stdin, termios.TIOCSTI, byte) kinda works to wake up a call to input on posix except openbsd. this could mix with the windows and ki_pending recipes.

python-trio / trio