Open graingert opened 3 weeks ago
this is because Cython coroutines appear to have a frame, so it doesn't get a wrapper python coroutine:
but don't actually push that frame when the coroutine runs
I think we need to check if not cr_frame.f_code.co_code:
but don't actually push that frame when the coroutine runs
Should this be raised as a Cython bug? (I know that doesn't help us in the meanwhile but having an attribute in some cases but not all is not useful)
I think we need to check
if not cr_frame.f_code.co_code:
I assume this would undo any sort of KI protection enabled by outer scopes, because it's impossible to know whether it's enabling or disabling KI protection? I think that makes sense, though maybe it would be better to detect that the frame will not be there and then create a wrapper, unless bad
gets a frame if Cython detects there's a decorator. This should presumably still block forever in Cython:
@enable_ki_protection
async def bad():
sleep()
We could have two async wrappers for cython, one protected and one unprotected, and use the KI protected-ness of the coro.cr_frame.co_code to pick which one you use
This would not work for cases like this:
@enable_ki_protection
async def bad():
sleep()
async def outer():
return await bad()
The other option is for enable_ki_protection to detect cython functions and wrap them with an extra (async, generator,asyncgenerater)function
I think detecting Cython functions and wrapping then would ultimately be simpler and that's better.
We could have two async wrappers for cython, one protected and one unprotected, and use the KI protected-ness of the coro.cr_frame.co_code to pick which one you use
I don't get what you mean by this, though I'm also tired so it could be that too.
OK so looking at this more, in general sys._getframe
doesn't get the frame for any given cython function, sync or async:
hello.pyx
:
import sys
def synctest():
print(sys._getframe(0))
def synctestwrapper():
synctest()
in a REPL:
>>> import hello
>>> hello.synctest()
<frame at 0x00000166F3D6A200, file '<stdin>', line 1, code <module>>
>>> hello.synctestwrapper()
<frame at 0x00000166F3D6A200, file '<stdin>', line 1, code <module>>
Looking at cython's limitations page, this is intentional:
Currently we generate fake tracebacks as part of exception propagation, but don’t fill in locals and can’t fill in co_code. To be fully compatible, we would have to generate these stack frame objects at function call time (with a potential performance penalty). We may have an option to enable this for debugging.
Given that, I think we should create a wrapper around things if inspect.isfunction
returns False
? I can't imagine builtin methods work with KI protection either...
That somewhat negates the benefits of using the __code__
for ki protection (because we'll still need the lines for wrapping generators vs normal functions vs whatever) but we should still have the speed benefits in the common case.
if you write a python function:
and call it from a cython task:
it hangs forever and can't be interrupted with ctrl+c
currently_ki_protected sees the following stack: