Closed nmlorg closed 1 month ago
Everything coming into the executor as ret
via ret = coro.send(...)
will be wrapped in an object that implements:
Wrapper.finalized
Wrapper.value
None
until finalized = True
).Wrapper.finalize(value)
finalized = True
and value = value
— then notify the object waiting on this awaitable that this awaitable has finalized (by calling its step()
).Wrapper.step()
the top-level: https://github.com/nmlorg/nsync/blob/f9b23887089618f6df42e69d19520fc0575b456a/examples/async5.py#L212-L214
will first create:
coro = CoroutineWrapper(…):
finalized = False
value = None
_coro = g()
_waiting_for = None
Then either the executor or the constructor will call self.step()
, which will execute coro_token_coro_token
up to await gathertoken
, changing the tree to:
coro = CoroutineWrapper(g()):
finalized = False
value = None
_coro = g()
_waiting_for = GatherWrapper(GatherToken(…)):
finalized = False
value = None
_deadline = None
_sockets = ()
_waiting_for = [
CoroutineWrapper(wait_for(…)):
finalized = False
value = None
_coro = wait_for(…)
_waiting_for = None
]
but (this is why I'm thinking the constructor will make the first step()
call) that will immediately execute wait_for(…).send(None)
, executing
https://github.com/nmlorg/nsync/blob/f9b23887089618f6df42e69d19520fc0575b456a/examples/async5.py#L67-L77
up to await token
, changing the tree to:
coro = CoroutineWrapper(g()):
finalized = False
value = None
_coro = g()
_waiting_for = GatherWrapper(GatherToken(…)):
finalized = False
value = None
_deadline = None
_sockets = ()
_waiting_for = [
CoroutineWrapper(wait_for(…)):
finalized = False
value = None
_coro = wait_for(…)
_waiting_for = SleepWrapper(SleepToken(…)):
finalized = False
value = None
_deadline = time.time() + .4
_sockets = ()
]
At this point, the executor will need to figure out what we're actually waiting for. It will call something like coro.waiting_for()
and get back (), time.time + .4
.
After the executor runs time.sleep(time.time() - time.time() + .4)
, it will somehow get a handle on the SleepWrapper
instance and call something like:
deadline_awaitable.finalize(None)
which will first change the tree to:
coro = CoroutineWrapper(g()):
finalized = False
value = None
_coro = g()
_waiting_for = GatherWrapper(GatherToken(…)):
finalized = False
value = None
_deadline = None
_sockets = ()
_waiting_for = [
CoroutineWrapper(wait_for(…)):
finalized = False
value = None
_coro = wait_for(…)
_waiting_for = SleepWrapper(SleepToken(…)):
finalized = True
value = None
_deadline = None
_sockets = ()
]
and then call step()
on its parent (the CoroutineWrapper
for wait_for
).
This will resume wait_for
at ret =
, then continue to return …
, which will raise StopIteration
, triggering the wrapper to finalize itself (self.finalize('wait_for final value')
, changing the tree to:
coro = CoroutineWrapper(g()):
finalized = False
value = None
_coro = g()
_waiting_for = GatherWrapper(GatherToken(…)):
finalized = False
value = None
_deadline = None
_sockets = ()
_waiting_for = [
CoroutineWrapper(wait_for(…)):
finalized = True
value = 'wait_for final value'
_coro = None
_waiting_for = None
]
before calling step()
on its parent (the GatherWrapper
).
This will iterate over everything it's waiting for (which is just the now-finalized CoroutineWrapper
), see that it's finished, and finalize itself (self.finalize(['wait_for final value'])
), changing the tree to:
coro = CoroutineWrapper(g()):
finalized = False
value = None
_coro = g()
_waiting_for = GatherWrapper(GatherToken(…)):
finalized = True
value = ['wait_for final value']
_deadline = None
_sockets = ()
_waiting_for = None
before calling step()
on its parent (the original CoroutineWrapper
).
This will resume coro_token_coro_token
at ret =
, then continue to return …
, which will raise StopIteration
, triggering the wrapper to finalize itself (self.finalize('coro_token_coro_token final value')
, changing the tree to:
coro = CoroutineWrapper(g()):
finalized = True
value = 'coro_token_coro_token final value'
_coro = g()
_waiting_for = None
before finally returning to main
.
(I think.)
0.002 [main:334] Coroutine waiting for a token:
0.002 [sync_await:282] coroutine = <coroutine object coro_token at 0x7648141582e0>
0.002 [wrap:170] parent = None awaitable = <coroutine object coro_token at 0x7648141582e0>
0.002 [__init__:194] self = <__main__.CoroutineWrapper object at 0x7648142bef50> parent = None coro = <coroutine object coro_token at 0x7648141582e0>
0.002 [_send:209] self = <__main__.CoroutineWrapper object at 0x7648142bef50> value = None
0.002 [coro_token:54] sleeptoken = SleepToken(.1):
0.002 [__init__:31] <__main__.SleepToken object at 0x7648142bf730> 0.1
0.002 [coro_token:56] sleeptoken = <__main__.SleepToken object at 0x7648142bf730>
0.003 [coro_token:58] ret = await sleeptoken:
0.003 [__await__:20] <__main__.SleepToken object at 0x7648142bf730>
0.003 [__await__:21] ret = yield self:
0.003 [wrap:170] parent = <__main__.CoroutineWrapper object at 0x7648142bef50> awaitable = <__main__.SleepToken object at 0x7648142bf730>
0.003 [__init__:239] self = <__main__.SleepWrapper object at 0x7648142bf4f0> parent = <__main__.CoroutineWrapper object at 0x7648142bef50> delay = 0.1
0.003 [get_waiting_for:218] self = <__main__.CoroutineWrapper object at 0x7648142bef50>
0.003 [get_waiting_for:244] self = <__main__.SleepWrapper object at 0x7648142bf4f0>
0.003 [__init__:185] self = <__main__._WaitingFor object at 0x7648142bf5b0> readers = () sleeper = <__main__.SleepWrapper object at 0x7648142bf4f0>
0.004 [sync_await:287] readers = () sleeper = <__main__.SleepWrapper object at 0x7648142bf4f0>
0.104 [finalize:150] self = <__main__.SleepWrapper object at 0x7648142bf4f0> value = None
0.105 [step:202] self = <__main__.CoroutineWrapper object at 0x7648142bef50>
0.105 [_send:209] self = <__main__.CoroutineWrapper object at 0x7648142bef50> value = None
0.105 [__await__:23] ret = None
0.105 [coro_token:60] ret = None
0.105 [coro_token:62] return 'coro_token final value'
0.105 [finalize:150] self = <__main__.CoroutineWrapper object at 0x7648142bef50> value = coro_token final value
0.105 [main:336] ret = 'coro_token final value'
0.105 [main:338] Coroutine waiting for a coroutine waiting for a token:
0.105 [sync_await:282] coroutine = <coroutine object coro_coro_token at 0x764814158350>
0.106 [wrap:170] parent = None awaitable = <coroutine object coro_coro_token at 0x764814158350>
0.106 [__init__:194] self = <__main__.CoroutineWrapper object at 0x7648142bf790> parent = None coro = <coroutine object coro_coro_token at 0x764814158350>
0.106 [_send:209] self = <__main__.CoroutineWrapper object at 0x7648142bf790> value = None
0.106 [coro_coro_token:82] sleeptoken = SleepToken(.2):
0.106 [__init__:31] <__main__.SleepToken object at 0x7648142bf460> 0.2
0.106 [coro_coro_token:84] sleeptoken = <__main__.SleepToken object at 0x7648142bf460>
0.106 [coro_coro_token:86] wait_for_coro = wait_for(sleeptoken):
0.106 [coro_coro_token:88] wait_for_coro = <coroutine object wait_for at 0x7648141583c0>
0.107 [coro_coro_token:90] ret = await wait_for_coro:
0.107 [wait_for:69] token = <__main__.SleepToken object at 0x7648142bf460>
0.107 [wait_for:71] ret = await token:
0.107 [__await__:20] <__main__.SleepToken object at 0x7648142bf460>
0.107 [__await__:21] ret = yield self:
0.107 [wrap:170] parent = <__main__.CoroutineWrapper object at 0x7648142bf790> awaitable = <__main__.SleepToken object at 0x7648142bf460>
0.107 [__init__:239] self = <__main__.SleepWrapper object at 0x7648142bf3a0> parent = <__main__.CoroutineWrapper object at 0x7648142bf790> delay = 0.2
0.107 [get_waiting_for:218] self = <__main__.CoroutineWrapper object at 0x7648142bf790>
0.108 [get_waiting_for:244] self = <__main__.SleepWrapper object at 0x7648142bf3a0>
0.108 [__init__:185] self = <__main__._WaitingFor object at 0x7648142bd330> readers = () sleeper = <__main__.SleepWrapper object at 0x7648142bf3a0>
0.108 [sync_await:287] readers = () sleeper = <__main__.SleepWrapper object at 0x7648142bf3a0>
0.308 [finalize:150] self = <__main__.SleepWrapper object at 0x7648142bf3a0> value = None
0.308 [step:202] self = <__main__.CoroutineWrapper object at 0x7648142bf790>
0.309 [_send:209] self = <__main__.CoroutineWrapper object at 0x7648142bf790> value = None
0.309 [__await__:23] ret = None
0.309 [wait_for:73] ret = None
0.309 [wait_for:75] return 'wait_for final value'
0.309 [coro_coro_token:92] ret = wait_for final value
0.309 [coro_coro_token:94] return 'coro_coro_token final value'
0.309 [finalize:150] self = <__main__.CoroutineWrapper object at 0x7648142bf790> value = coro_coro_token final value
0.310 [main:340] ret = 'coro_coro_token final value'
0.310 [main:342] Coroutine waiting for a token containing a token:
0.310 [sync_await:282] coroutine = <coroutine object coro_token_token at 0x764814158430>
0.310 [wrap:170] parent = None awaitable = <coroutine object coro_token_token at 0x764814158430>
0.310 [__init__:194] self = <__main__.CoroutineWrapper object at 0x7648142bf3a0> parent = None coro = <coroutine object coro_token_token at 0x764814158430>
0.310 [_send:209] self = <__main__.CoroutineWrapper object at 0x7648142bf3a0> value = None
0.310 [coro_token_token:101] sleeptoken = SleepToken(.3):
0.310 [__init__:31] <__main__.SleepToken object at 0x7648142bf760> 0.3
0.310 [coro_token_token:103] sleeptoken = <__main__.SleepToken object at 0x7648142bf760>
0.310 [coro_token_token:105] gathertoken = GatherToken(sleeptoken):
0.311 [__init__:47] <__main__.GatherToken object at 0x7648142bf790> (<__main__.SleepToken object at 0x7648142bf760>,)
0.311 [coro_token_token:107] gathertoken = <__main__.GatherToken object at 0x7648142bf790>
0.311 [coro_token_token:109] ret = await gathertoken:
0.311 [__await__:20] <__main__.GatherToken object at 0x7648142bf790>
0.311 [__await__:21] ret = yield self:
0.311 [wrap:170] parent = <__main__.CoroutineWrapper object at 0x7648142bf3a0> awaitable = <__main__.GatherToken object at 0x7648142bf790>
0.311 [__init__:252] self = <__main__.GatherWrapper object at 0x7648142bf2e0> parent = <__main__.CoroutineWrapper object at 0x7648142bf3a0> awaitables = (<__main__.SleepToken object at 0x7648142bf760>,)
0.312 [wrap:170] parent = <__main__.GatherWrapper object at 0x7648142bf2e0> awaitable = <__main__.SleepToken object at 0x7648142bf760>
0.312 [__init__:239] self = <__main__.SleepWrapper object at 0x7648142be6b0> parent = <__main__.GatherWrapper object at 0x7648142bf2e0> delay = 0.3
0.312 [get_waiting_for:218] self = <__main__.CoroutineWrapper object at 0x7648142bf3a0>
0.312 [get_waiting_for:267] self = <__main__.GatherWrapper object at 0x7648142bf2e0>
0.312 [get_waiting_for:244] self = <__main__.SleepWrapper object at 0x7648142be6b0>
0.312 [__init__:185] self = <__main__._WaitingFor object at 0x7648142bf310> readers = () sleeper = <__main__.SleepWrapper object at 0x7648142be6b0>
0.312 [__init__:185] self = <__main__._WaitingFor object at 0x7648142be7a0> readers = [] sleeper = <__main__.SleepWrapper object at 0x7648142be6b0>
0.312 [sync_await:287] readers = [] sleeper = <__main__.SleepWrapper object at 0x7648142be6b0>
0.613 [finalize:150] self = <__main__.SleepWrapper object at 0x7648142be6b0> value = None
0.613 [step:257] self = <__main__.GatherWrapper object at 0x7648142bf2e0>
0.614 [finalize:150] self = <__main__.GatherWrapper object at 0x7648142bf2e0> value = [None]
0.614 [step:202] self = <__main__.CoroutineWrapper object at 0x7648142bf3a0>
0.614 [_send:209] self = <__main__.CoroutineWrapper object at 0x7648142bf3a0> value = [None]
0.614 [__await__:23] ret = [None]
0.614 [coro_token_token:111] ret = [None]
0.614 [coro_token_token:113] return 'coro_token_token final value'
0.615 [finalize:150] self = <__main__.CoroutineWrapper object at 0x7648142bf3a0> value = coro_token_token final value
0.615 [main:344] ret = 'coro_token_token final value'
0.615 [main:346] Coroutine waiting for a token containing a coroutine waiting for a token:
0.615 [sync_await:282] coroutine = <coroutine object coro_token_coro_token at 0x764814158350>
0.615 [wrap:170] parent = None awaitable = <coroutine object coro_token_coro_token at 0x764814158350>
0.615 [__init__:194] self = <__main__.CoroutineWrapper object at 0x7648142bea70> parent = None coro = <coroutine object coro_token_coro_token at 0x764814158350>
0.615 [_send:209] self = <__main__.CoroutineWrapper object at 0x7648142bea70> value = None
0.615 [coro_token_coro_token:120] sleeptoken = SleepToken(.4):
0.615 [__init__:31] <__main__.SleepToken object at 0x7648142be8c0> 0.4
0.615 [coro_token_coro_token:122] sleeptoken = <__main__.SleepToken object at 0x7648142be8c0>
0.616 [coro_token_coro_token:124] wait_for_coro = wait_for(sleeptoken):
0.616 [coro_token_coro_token:126] wait_for_coro = <coroutine object wait_for at 0x764814158510>
0.616 [coro_token_coro_token:128] gathertoken = GatherToken(wait_for_coro):
0.616 [__init__:47] <__main__.GatherToken object at 0x7648142beec0> (<coroutine object wait_for at 0x764814158510>,)
0.617 [coro_token_coro_token:130] gathertoken = <__main__.GatherToken object at 0x7648142beec0>
0.617 [coro_token_coro_token:132] ret = await gathertoken:
0.617 [__await__:20] <__main__.GatherToken object at 0x7648142beec0>
0.617 [__await__:21] ret = yield self:
0.617 [wrap:170] parent = <__main__.CoroutineWrapper object at 0x7648142bea70> awaitable = <__main__.GatherToken object at 0x7648142beec0>
0.617 [__init__:252] self = <__main__.GatherWrapper object at 0x7648142bf3a0> parent = <__main__.CoroutineWrapper object at 0x7648142bea70> awaitables = (<coroutine object wait_for at 0x764814158510>,)
0.617 [wrap:170] parent = <__main__.GatherWrapper object at 0x7648142bf3a0> awaitable = <coroutine object wait_for at 0x764814158510>
0.618 [__init__:194] self = <__main__.CoroutineWrapper object at 0x7648142be950> parent = <__main__.GatherWrapper object at 0x7648142bf3a0> coro = <coroutine object wait_for at 0x764814158510>
0.618 [_send:209] self = <__main__.CoroutineWrapper object at 0x7648142be950> value = None
0.618 [wait_for:69] token = <__main__.SleepToken object at 0x7648142be8c0>
0.618 [wait_for:71] ret = await token:
0.618 [__await__:20] <__main__.SleepToken object at 0x7648142be8c0>
0.619 [__await__:21] ret = yield self:
0.619 [wrap:170] parent = <__main__.CoroutineWrapper object at 0x7648142be950> awaitable = <__main__.SleepToken object at 0x7648142be8c0>
0.619 [__init__:239] self = <__main__.SleepWrapper object at 0x7648142bf400> parent = <__main__.CoroutineWrapper object at 0x7648142be950> delay = 0.4
0.619 [get_waiting_for:218] self = <__main__.CoroutineWrapper object at 0x7648142bea70>
0.619 [get_waiting_for:267] self = <__main__.GatherWrapper object at 0x7648142bf3a0>
0.619 [get_waiting_for:218] self = <__main__.CoroutineWrapper object at 0x7648142be950>
0.619 [get_waiting_for:244] self = <__main__.SleepWrapper object at 0x7648142bf400>
0.620 [__init__:185] self = <__main__._WaitingFor object at 0x7648142bf760> readers = () sleeper = <__main__.SleepWrapper object at 0x7648142bf400>
0.620 [__init__:185] self = <__main__._WaitingFor object at 0x7648142bf790> readers = [] sleeper = <__main__.SleepWrapper object at 0x7648142bf400>
0.620 [sync_await:287] readers = [] sleeper = <__main__.SleepWrapper object at 0x7648142bf400>
1.020 [finalize:150] self = <__main__.SleepWrapper object at 0x7648142bf400> value = None
1.020 [step:202] self = <__main__.CoroutineWrapper object at 0x7648142be950>
1.021 [_send:209] self = <__main__.CoroutineWrapper object at 0x7648142be950> value = None
1.021 [__await__:23] ret = None
1.021 [wait_for:73] ret = None
1.021 [wait_for:75] return 'wait_for final value'
1.021 [finalize:150] self = <__main__.CoroutineWrapper object at 0x7648142be950> value = wait_for final value
1.021 [step:257] self = <__main__.GatherWrapper object at 0x7648142bf3a0>
1.021 [finalize:150] self = <__main__.GatherWrapper object at 0x7648142bf3a0> value = ['wait_for final value']
1.022 [step:202] self = <__main__.CoroutineWrapper object at 0x7648142bea70>
1.022 [_send:209] self = <__main__.CoroutineWrapper object at 0x7648142bea70> value = ['wait_for final value']
1.022 [__await__:23] ret = ['wait_for final value']
1.022 [coro_token_coro_token:134] ret = ['wait_for final value']
1.022 [coro_token_coro_token:136] return 'coro_token_coro_token final value'
1.023 [finalize:150] self = <__main__.CoroutineWrapper object at 0x7648142bea70> value = coro_token_coro_token final value
1.023 [main:348] ret = 'coro_token_coro_token final value'
0.002 [main:290] ret = sync_await(async_read()):
0.002 [sync_await:251] coroutine = <coroutine object async_read at 0x747185f60b30>
0.002 [wrap:133] parent = None awaitable = <coroutine object async_read at 0x747185f60b30>
0.002 [__init__:157] self = <__main__.CoroutineWrapper object at 0x747185f69780> parent = None coro = <coroutine object async_read at 0x747185f60b30>
0.002 [_send:172] self = <__main__.CoroutineWrapper object at 0x747185f69780> value = None
0.002 [async_read:55] sleep1 = SleepToken(.1):
0.002 [__init__:32] <__main__.SleepToken object at 0x747185f69180> 0.1
0.002 [async_read:57] sleep1 = <__main__.SleepToken object at 0x747185f69180>
0.003 [async_read:59] sleep2 = SleepToken(1.5):
0.003 [__init__:32] <__main__.SleepToken object at 0x747185f692a0> 1.5
0.003 [async_read:61] sleep2 = <__main__.SleepToken object at 0x747185f692a0>
0.003 [async_read:63] sleep3 = SleepToken(1.5):
0.003 [__init__:32] <__main__.SleepToken object at 0x747185f69210> 1.5
0.003 [async_read:65] sleep3 = <__main__.SleepToken object at 0x747185f69210>
0.003 [async_read:67] sleep4 = SleepToken(2.5):
0.003 [__init__:32] <__main__.SleepToken object at 0x747185f69360> 2.5
0.004 [async_read:69] sleep4 = <__main__.SleepToken object at 0x747185f69360>
0.004 [async_read:71] sock1 = socket.create_connection:
0.347 [async_read:74] sock1.sendall:
0.347 [async_read:77] read1 = ReadToken(sock1):
0.347 [__init__:40] <__main__.ReadToken object at 0x747185f691e0> <socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('xxx', 58866), raddr=('44.214.66.106', 80)>
0.347 [async_read:79] read1 = <__main__.ReadToken object at 0x747185f691e0>
0.348 [async_read:81] sock2 = socket.create_connection:
0.556 [async_read:84] sock2.sendall:
0.556 [async_read:87] read2 = ReadToken(sock2):
0.556 [__init__:40] <__main__.ReadToken object at 0x747185f696c0> <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('xxx', 46288), raddr=('54.159.225.35', 80)>
0.557 [async_read:89] read2 = <__main__.ReadToken object at 0x747185f696c0>
0.557 [async_read:91] gathertoken = GatherToken(sleep1, sleep2, sleep3, sleep4, read1, sleep2):
0.557 [__init__:48] <__main__.GatherToken object at 0x747185f69e10> (<__main__.SleepToken object at 0x747185f69180>, <__main__.SleepToken object at 0x747185f692a0>, <__main__.SleepToken object at 0x747185f69210>, <__main__.SleepToken object at 0x747185f69360>, <__main__.ReadToken object at 0x747185f691e0>, <__main__.ReadToken object at 0x747185f696c0>)
0.557 [async_read:93] gathertoken = <__main__.GatherToken object at 0x747185f69e10>
0.557 [async_read:95] ret = await gathertoken
0.557 [__await__:21] <__main__.GatherToken object at 0x747185f69e10>
0.558 [__await__:22] ret = yield self:
0.558 [wrap:133] parent = <__main__.CoroutineWrapper object at 0x747185f69780> awaitable = <__main__.GatherToken object at 0x747185f69e10>
0.558 [__init__:218] self = <__main__.GatherWrapper object at 0x747185f69f30> parent = <__main__.CoroutineWrapper object at 0x747185f69780> awaitables = (<__main__.SleepToken object at 0x747185f69180>, <__main__.SleepToken object at 0x747185f692a0>, <__main__.SleepToken object at 0x747185f69210>, <__main__.SleepToken object at 0x747185f69360>, <__main__.ReadToken object at 0x747185f691e0>, <__main__.ReadToken object at 0x747185f696c0>)
0.558 [wrap:133] parent = <__main__.GatherWrapper object at 0x747185f69f30> awaitable = <__main__.SleepToken object at 0x747185f69180>
0.559 [__init__:204] self = <__main__.SleepWrapper object at 0x747185f69fc0> parent = <__main__.GatherWrapper object at 0x747185f69f30> delay = 0.1
0.559 [wrap:133] parent = <__main__.GatherWrapper object at 0x747185f69f30> awaitable = <__main__.SleepToken object at 0x747185f692a0>
0.559 [__init__:204] self = <__main__.SleepWrapper object at 0x747185f6a020> parent = <__main__.GatherWrapper object at 0x747185f69f30> delay = 1.5
0.559 [wrap:133] parent = <__main__.GatherWrapper object at 0x747185f69f30> awaitable = <__main__.SleepToken object at 0x747185f69210>
0.559 [__init__:204] self = <__main__.SleepWrapper object at 0x747185f6a080> parent = <__main__.GatherWrapper object at 0x747185f69f30> delay = 1.5
0.560 [wrap:133] parent = <__main__.GatherWrapper object at 0x747185f69f30> awaitable = <__main__.SleepToken object at 0x747185f69360>
0.560 [__init__:204] self = <__main__.SleepWrapper object at 0x747185f6a0e0> parent = <__main__.GatherWrapper object at 0x747185f69f30> delay = 2.5
0.560 [wrap:133] parent = <__main__.GatherWrapper object at 0x747185f69f30> awaitable = <__main__.ReadToken object at 0x747185f691e0>
0.560 [__init__:190] self = <__main__.ReadWrapper object at 0x747185f6a140> parent = <__main__.GatherWrapper object at 0x747185f69f30> sock = <socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('xxx', 58866), raddr=('44.214.66.106', 80)>
0.560 [wrap:133] parent = <__main__.GatherWrapper object at 0x747185f69f30> awaitable = <__main__.ReadToken object at 0x747185f696c0>
0.560 [__init__:190] self = <__main__.ReadWrapper object at 0x747185f6a1a0> parent = <__main__.GatherWrapper object at 0x747185f69f30> sock = <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('xxx', 46288), raddr=('54.159.225.35', 80)>
0.561 [get_waiting_for:181] self = <__main__.CoroutineWrapper object at 0x747185f69780>
0.561 [get_waiting_for:233] self = <__main__.GatherWrapper object at 0x747185f69f30>
0.561 [get_waiting_for:209] self = <__main__.SleepWrapper object at 0x747185f69fc0>
0.561 [__init__:148] self = <__main__._WaitingFor object at 0x747185f69e70> readers = () sleeper = <__main__.SleepWrapper object at 0x747185f69fc0>
0.561 [get_waiting_for:209] self = <__main__.SleepWrapper object at 0x747185f6a020>
0.561 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a290> readers = () sleeper = <__main__.SleepWrapper object at 0x747185f6a020>
0.561 [get_waiting_for:209] self = <__main__.SleepWrapper object at 0x747185f6a080>
0.561 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a2f0> readers = () sleeper = <__main__.SleepWrapper object at 0x747185f6a080>
0.561 [get_waiting_for:209] self = <__main__.SleepWrapper object at 0x747185f6a0e0>
0.562 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a350> readers = () sleeper = <__main__.SleepWrapper object at 0x747185f6a0e0>
0.562 [get_waiting_for:195] self = <__main__.ReadWrapper object at 0x747185f6a140>
0.562 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a3b0> readers = [<__main__.ReadWrapper object at 0x747185f6a140>] sleeper = None
0.562 [get_waiting_for:195] self = <__main__.ReadWrapper object at 0x747185f6a1a0>
0.562 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a410> readers = [<__main__.ReadWrapper object at 0x747185f6a1a0>] sleeper = None
0.562 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a200> readers = [<__main__.ReadWrapper object at 0x747185f6a140>, <__main__.ReadWrapper object at 0x747185f6a1a0>] sleeper = <__main__.SleepWrapper object at 0x747185f69fc0>
0.562 [process_awaitables:263] readers = [<__main__.ReadWrapper object at 0x747185f6a140>, <__main__.ReadWrapper object at 0x747185f6a1a0>] sleeper = <__main__.SleepWrapper object at 0x747185f69fc0>
0.562 [process_awaitables:276] rlist = select.select rlist = [<socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('xxx', 58866), raddr=('44.214.66.106', 80)>, <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('xxx', 46288), raddr=('54.159.225.35', 80)>] timeout = 0.09648919105529785
0.660 [process_awaitables:278] rlist = []
0.660 [finalize:113] self = <__main__.SleepWrapper object at 0x747185f69fc0> value = select.select timed out after 0.09648919105529785 s
0.660 [step:223] self = <__main__.GatherWrapper object at 0x747185f69f30>
0.660 [get_waiting_for:181] self = <__main__.CoroutineWrapper object at 0x747185f69780>
0.661 [get_waiting_for:233] self = <__main__.GatherWrapper object at 0x747185f69f30>
0.661 [get_waiting_for:209] self = <__main__.SleepWrapper object at 0x747185f6a020>
0.661 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a4a0> readers = () sleeper = <__main__.SleepWrapper object at 0x747185f6a020>
0.661 [get_waiting_for:209] self = <__main__.SleepWrapper object at 0x747185f6a080>
0.661 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a410> readers = () sleeper = <__main__.SleepWrapper object at 0x747185f6a080>
0.661 [get_waiting_for:209] self = <__main__.SleepWrapper object at 0x747185f6a0e0>
0.661 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a3b0> readers = () sleeper = <__main__.SleepWrapper object at 0x747185f6a0e0>
0.661 [get_waiting_for:195] self = <__main__.ReadWrapper object at 0x747185f6a140>
0.662 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a350> readers = [<__main__.ReadWrapper object at 0x747185f6a140>] sleeper = None
0.662 [get_waiting_for:195] self = <__main__.ReadWrapper object at 0x747185f6a1a0>
0.662 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a2f0> readers = [<__main__.ReadWrapper object at 0x747185f6a1a0>] sleeper = None
0.662 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a230> readers = [<__main__.ReadWrapper object at 0x747185f6a140>, <__main__.ReadWrapper object at 0x747185f6a1a0>] sleeper = <__main__.SleepWrapper object at 0x747185f6a020>
0.662 [process_awaitables:263] readers = [<__main__.ReadWrapper object at 0x747185f6a140>, <__main__.ReadWrapper object at 0x747185f6a1a0>] sleeper = <__main__.SleepWrapper object at 0x747185f6a020>
0.662 [process_awaitables:276] rlist = select.select rlist = [<socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('xxx', 58866), raddr=('44.214.66.106', 80)>, <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('xxx', 46288), raddr=('54.159.225.35', 80)>] timeout = 1.3970942497253418
0.812 [process_awaitables:278] rlist = [<socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('xxx', 58866), raddr=('44.214.66.106', 80)>]
0.812 [finalize:113] self = <__main__.ReadWrapper object at 0x747185f6a140> value = b'HTTP/1.1 200 OK\r\nDate: Wed, 25 Sep 2024 18:28:05 GMT\r\nContent-Type: text/html; charset=utf-8\r\nContent-Length: 0\r\nConnection: close\r\nServer: gunicorn/19.9.0\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Credentials: true\r\n\r\n'
0.813 [step:223] self = <__main__.GatherWrapper object at 0x747185f69f30>
0.813 [get_waiting_for:181] self = <__main__.CoroutineWrapper object at 0x747185f69780>
0.813 [get_waiting_for:233] self = <__main__.GatherWrapper object at 0x747185f69f30>
0.813 [get_waiting_for:209] self = <__main__.SleepWrapper object at 0x747185f6a020>
0.813 [__init__:148] self = <__main__._WaitingFor object at 0x747185f69e40> readers = () sleeper = <__main__.SleepWrapper object at 0x747185f6a020>
0.813 [get_waiting_for:209] self = <__main__.SleepWrapper object at 0x747185f6a080>
0.813 [__init__:148] self = <__main__._WaitingFor object at 0x747185f695d0> readers = () sleeper = <__main__.SleepWrapper object at 0x747185f6a080>
0.814 [get_waiting_for:209] self = <__main__.SleepWrapper object at 0x747185f6a0e0>
0.814 [__init__:148] self = <__main__._WaitingFor object at 0x747185f69f00> readers = () sleeper = <__main__.SleepWrapper object at 0x747185f6a0e0>
0.814 [get_waiting_for:195] self = <__main__.ReadWrapper object at 0x747185f6a1a0>
0.814 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a4d0> readers = [<__main__.ReadWrapper object at 0x747185f6a1a0>] sleeper = None
0.814 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a470> readers = [<__main__.ReadWrapper object at 0x747185f6a1a0>] sleeper = <__main__.SleepWrapper object at 0x747185f6a020>
0.814 [process_awaitables:263] readers = [<__main__.ReadWrapper object at 0x747185f6a1a0>] sleeper = <__main__.SleepWrapper object at 0x747185f6a020>
0.814 [process_awaitables:276] rlist = select.select rlist = [<socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('xxx', 46288), raddr=('54.159.225.35', 80)>] timeout = 1.2449710369110107
2.061 [process_awaitables:278] rlist = []
2.061 [finalize:113] self = <__main__.SleepWrapper object at 0x747185f6a020> value = select.select timed out after 1.2449710369110107 s
2.062 [step:223] self = <__main__.GatherWrapper object at 0x747185f69f30>
2.062 [get_waiting_for:181] self = <__main__.CoroutineWrapper object at 0x747185f69780>
2.062 [get_waiting_for:233] self = <__main__.GatherWrapper object at 0x747185f69f30>
2.062 [get_waiting_for:209] self = <__main__.SleepWrapper object at 0x747185f6a080>
2.062 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a5c0> readers = () sleeper = <__main__.SleepWrapper object at 0x747185f6a080>
2.062 [get_waiting_for:209] self = <__main__.SleepWrapper object at 0x747185f6a0e0>
2.062 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a650> readers = () sleeper = <__main__.SleepWrapper object at 0x747185f6a0e0>
2.062 [get_waiting_for:195] self = <__main__.ReadWrapper object at 0x747185f6a1a0>
2.062 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a6b0> readers = [<__main__.ReadWrapper object at 0x747185f6a1a0>] sleeper = None
2.063 [__init__:148] self = <__main__._WaitingFor object at 0x747185f69e70> readers = [<__main__.ReadWrapper object at 0x747185f6a1a0>] sleeper = <__main__.SleepWrapper object at 0x747185f6a080>
2.063 [process_awaitables:263] readers = [<__main__.ReadWrapper object at 0x747185f6a1a0>] sleeper = <__main__.SleepWrapper object at 0x747185f6a080>
2.063 [finalize:113] self = <__main__.SleepWrapper object at 0x747185f6a080> value = deadline (1727288886.3983293) <= now (1727288886.401572)
2.063 [step:223] self = <__main__.GatherWrapper object at 0x747185f69f30>
2.063 [get_waiting_for:181] self = <__main__.CoroutineWrapper object at 0x747185f69780>
2.063 [get_waiting_for:233] self = <__main__.GatherWrapper object at 0x747185f69f30>
2.063 [get_waiting_for:209] self = <__main__.SleepWrapper object at 0x747185f6a0e0>
2.063 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a5f0> readers = () sleeper = <__main__.SleepWrapper object at 0x747185f6a0e0>
2.063 [get_waiting_for:195] self = <__main__.ReadWrapper object at 0x747185f6a1a0>
2.064 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a7d0> readers = [<__main__.ReadWrapper object at 0x747185f6a1a0>] sleeper = None
2.064 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a770> readers = [<__main__.ReadWrapper object at 0x747185f6a1a0>] sleeper = <__main__.SleepWrapper object at 0x747185f6a0e0>
2.064 [process_awaitables:263] readers = [<__main__.ReadWrapper object at 0x747185f6a1a0>] sleeper = <__main__.SleepWrapper object at 0x747185f6a0e0>
2.064 [process_awaitables:276] rlist = select.select rlist = [<socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('xxx', 46288), raddr=('54.159.225.35', 80)>] timeout = 0.9959819316864014
3.014 [process_awaitables:278] rlist = [<socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('xxx', 46288), raddr=('54.159.225.35', 80)>]
3.014 [finalize:113] self = <__main__.ReadWrapper object at 0x747185f6a1a0> value = b'HTTP/1.1 200 OK\r\nDate: Wed, 25 Sep 2024 18:28:07 GMT\r\nContent-Type: text/html; charset=utf-8\r\nContent-Length: 0\r\nConnection: close\r\nServer: gunicorn/19.9.0\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Credentials: true\r\n\r\n'
3.014 [step:223] self = <__main__.GatherWrapper object at 0x747185f69f30>
3.014 [get_waiting_for:181] self = <__main__.CoroutineWrapper object at 0x747185f69780>
3.014 [get_waiting_for:233] self = <__main__.GatherWrapper object at 0x747185f69f30>
3.015 [get_waiting_for:209] self = <__main__.SleepWrapper object at 0x747185f6a0e0>
3.015 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a5f0> readers = () sleeper = <__main__.SleepWrapper object at 0x747185f6a0e0>
3.015 [__init__:148] self = <__main__._WaitingFor object at 0x747185f6a6b0> readers = [] sleeper = <__main__.SleepWrapper object at 0x747185f6a0e0>
3.015 [process_awaitables:263] readers = [] sleeper = <__main__.SleepWrapper object at 0x747185f6a0e0>
3.015 [process_awaitables:276] rlist = select.select rlist = [] timeout = 0.044820308685302734
3.061 [process_awaitables:278] rlist = []
3.061 [finalize:113] self = <__main__.SleepWrapper object at 0x747185f6a0e0> value = select.select timed out after 0.044820308685302734 s
3.061 [step:223] self = <__main__.GatherWrapper object at 0x747185f69f30>
3.061 [finalize:113] self = <__main__.GatherWrapper object at 0x747185f69f30> value = ['select.select timed out after 0.09648919105529785 s', 'select.select timed out after 1.2449710369110107 s', 'deadline (1727288886.3983293) <= now (1727288886.401572)', 'select.select timed out after 0.044820308685302734 s', b'HTTP/1.1 200 OK\r\nDate: Wed, 25 Sep 2024 18:28:05 GMT\r\nContent-Type: text/html; charset=utf-8\r\nContent-Length: 0\r\nConnection: close\r\nServer: gunicorn/19.9.0\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Credentials: true\r\n\r\n', b'HTTP/1.1 200 OK\r\nDate: Wed, 25 Sep 2024 18:28:07 GMT\r\nContent-Type: text/html; charset=utf-8\r\nContent-Length: 0\r\nConnection: close\r\nServer: gunicorn/19.9.0\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Credentials: true\r\n\r\n']
3.061 [step:165] self = <__main__.CoroutineWrapper object at 0x747185f69780>
3.062 [_send:172] self = <__main__.CoroutineWrapper object at 0x747185f69780> value = ['select.select timed out after 0.09648919105529785 s', 'select.select timed out after 1.2449710369110107 s', 'deadline (1727288886.3983293) <= now (1727288886.401572)', 'select.select timed out after 0.044820308685302734 s', b'HTTP/1.1 200 OK\r\nDate: Wed, 25 Sep 2024 18:28:05 GMT\r\nContent-Type: text/html; charset=utf-8\r\nContent-Length: 0\r\nConnection: close\r\nServer: gunicorn/19.9.0\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Credentials: true\r\n\r\n', b'HTTP/1.1 200 OK\r\nDate: Wed, 25 Sep 2024 18:28:07 GMT\r\nContent-Type: text/html; charset=utf-8\r\nContent-Length: 0\r\nConnection: close\r\nServer: gunicorn/19.9.0\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Credentials: true\r\n\r\n']
3.062 [__await__:24] ret = ['select.select timed out after 0.09648919105529785 s', 'select.select timed out after 1.2449710369110107 s', 'deadline (1727288886.3983293) <= now (1727288886.401572)', 'select.select timed out after 0.044820308685302734 s', b'HTTP/1.1 200 OK\r\nDate: Wed, 25 Sep 2024 18:28:05 GMT\r\nContent-Type: text/html; charset=utf-8\r\nContent-Length: 0\r\nConnection: close\r\nServer: gunicorn/19.9.0\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Credentials: true\r\n\r\n', b'HTTP/1.1 200 OK\r\nDate: Wed, 25 Sep 2024 18:28:07 GMT\r\nContent-Type: text/html; charset=utf-8\r\nContent-Length: 0\r\nConnection: close\r\nServer: gunicorn/19.9.0\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Credentials: true\r\n\r\n']
3.062 [async_read:97] ret = ['select.select timed out after 0.09648919105529785 s', 'select.select timed out after 1.2449710369110107 s', 'deadline (1727288886.3983293) <= now (1727288886.401572)', 'select.select timed out after 0.044820308685302734 s', b'HTTP/1.1 200 OK\r\nDate: Wed, 25 Sep 2024 18:28:05 GMT\r\nContent-Type: text/html; charset=utf-8\r\nContent-Length: 0\r\nConnection: close\r\nServer: gunicorn/19.9.0\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Credentials: true\r\n\r\n', b'HTTP/1.1 200 OK\r\nDate: Wed, 25 Sep 2024 18:28:07 GMT\r\nContent-Type: text/html; charset=utf-8\r\nContent-Length: 0\r\nConnection: close\r\nServer: gunicorn/19.9.0\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Credentials: true\r\n\r\n']
3.062 [async_read:99] return 'async_read done!'
3.062 [finalize:113] self = <__main__.CoroutineWrapper object at 0x747185f69780> value = async_read done!
3.062 [main:292] ret = async_read done!
Anything that can be passed to the
await
operator (1. coroutines and 2. objects that implement an__await__
generator method) is an awaitable.All awaitables are always in one of 5 states:
The code:
creates an asynchronous function
f
.creates a coroutine instance
coro
in state 1 (waiting to be run).moves the coroutine into state 2 (running); it immediately evaluates:
and then returns whatever
awaitable.__await__()
yields asret
, moving the coroutine into state 3 (waiting for another awaitable).Whatever called
coro.send
(the executor) is now responsible for finalizingawaitable1
. This could be waiting for and reading data from the network, waiting for the system clock to hit a specific time, etc. Onceawaitable1
finalizes, the executor is responsible for feeding the finalized value back into the coroutine instance by callingcoro.send(awaitable1's finalized value)
.This causes the coroutine to move back to state 2, where it evaluates:
At this point the coroutine moves back to state 3, waiting for the executor to finalize
awaitable2
and feed its finalized value back by callingcoro.send(awaitable2's finalized value)
, moving the coroutine back to state 2, where it evaluates:This raises
StopExecution(value=value3)
, which tells the executor that the coroutine is finalized (and its final value isvalue3
).At this point, if any coroutine instance was in state 3 waiting for
coro
, the executor would feedvalue3
(coro
's finalized value) into that parent coroutine viaparentcoro.send(value3)
.For what will be examples/async5.py, I need to implement some kind of recursive executor to handle the situation of a coroutine waiting for a token waiting for a coroutine.
I'm thinking of creating a wrapper (or set of wrappers) that provides a single unified
wrapper.finalized
,wrapper.value
, and something likewrapper.sockets
andwrapper.deadline
— some API to access not just the socket and/or deadline that that awaitable is directly waiting for, but all sockets (and the earliest deadline) for every awaitable "behind" it.I keep getting really, really turned around in my head every time I try to build this, but I think the executor (
sync_await
) will end up being something like:where
awaitable.finalize
will just:for tokens, but will recurse into itself for coroutine wrappers somehow (telling their tokens to finalize, and optionally calling
self.send
if that moved it from state 3 to 1) — so some ofsync_await
will probably end up inBaseWrapper.finalize
.