Tutorial introduction example (with a change)

[skgbanga]

Curio Tutorial uses the following example to introduce some of its concepts.

# hello.py                                                                                                                                                                                                                                                                                                                                                  
import curio                                                                                                                                                                                                                                                                                                                                                

async def friend(name):                                                                                                                                                                                                                                                                                                                                     
    print('Hi, my name is', name)                                                                                                                                                                                                                                                                                                                           
    print('Playing Minecraft')                                                                                                                                                                                                                                                                                                                              
    try:                                                                                                                                                                                                                                                                                                                                                    
        await curio.sleep(1000)                                                                                                                                                                                                                                                                                                                             
    except curio.CancelledError:                                                                                                                                                                                                                                                                                                                            
        print(name, 'going home')                                                                                                                                                                                                                                                                                                                           
        raise                                                                                                                                                                                                                                                                                                                                               

async def countdown(n):                                                                                                                                                                                                                                                                                                                                     
    while n > 0:                                                                                                                                                                                                                                                                                                                                            
        print('T-minus', n)                                                                                                                                                                                                                                                                                                                                 
        await curio.sleep(1)                                                                                                                                                                                                                                                                                                                                
        n -= 1                                                                                                                                                                                                                                                                                                                                              

async def kid():                                                                                                                                                                                                                                                                                                                                            
    print('Building the Millenium Falcon in Minecraft')                                                                                                                                                                                                                                                                                                     

    async with curio.TaskGroup() as f:                                                                                                                                                                                                                                                                                                                      
        await f.spawn(friend, 'Max')                                                                                                                                                                                                                                                                                                                        
        await f.spawn(friend, 'Lillian')                                                                                                                                                                                                                                                                                                                    
        await f.spawn(friend, 'Thomas')                                                                                                                                                                                                                                                                                                                     

    try:                                                                                                                                                                                                                                                                                                                                                    
        await curio.sleep(1000)                                                                                                                                                                                                                                                                                                                             
    except curio.CancelledError:                                                                                                                                                                                                                                                                                                                            
        print('Fine. Saving my work.')                                                                                                                                                                                                                                                                                                                      
        raise                                                                                                                                                                                                                                                                                                                                               

async def parent():                                                                                                                                                                                                                                                                                                                                         
    kid_task = await curio.spawn(kid)                                                                                                                                                                                                                                                                                                                       
    await curio.sleep(5)                                                                                                                                                                                                                                                                                                                                    

    print("Let's go")                                                                                                                                                                                                                                                                                                                                       
    count_task = await curio.spawn(countdown, 10)                                                                                                                                                                                                                                                                                                           
    await count_task.join()                                                                                                                                                                                                                                                                                                                                 

    print("We're leaving!")                                                                                                                                                                                                                                                                                                                                 
    try:                                                                                                                                                                                                                                                                                                                                                    
        await curio.timeout_after(10, kid_task.join)                                                                                                                                                                                                                                                                                                        
    except curio.TaskTimeout:                                                                                                                                                                                                                                                                                                                               
        print('I warned you!')                                                                                                                                                                                                                                                                                                                              
        await kid_task.cancel()                                                                                                                                                                                                                                                                                                                             
    print('Leaving!')                                                                                                                                                                                                                                                                                                                                       

if __name__ == '__main__':                                                                                                                                                                                                                                                                                                                                  
    curio.run(parent)

The above code is not exactly similar to the example mentioned on the page, kid() function has one change. try/except block is outside the async with block.

The above example produces the following output:

Building the Millenium Falcon in Minecraft
Hi, my name is Max
Playing Minecraft
Hi, my name is Lillian
Playing Minecraft
Hi, my name is Thomas
Playing Minecraft
Let's go
T-minus 10
T-minus 9
T-minus 8
T-minus 7
T-minus 6
T-minus 5
T-minus 4
T-minus 3
T-minus 2
T-minus 1
We're leaving!
I warned you!
Max going home
Thomas going home
Lillian going home
Leaving!

As you can see, the line Fine. Saving my work. never gets printed. (All the friends got the CancelledError exception and printed ... going home fine.

Is this intential behaviour? If yes, this is slighly surprising.

[skgbanga]

Ah, the kid never really goes to the TIME_SLEEP state.

curio > ps  
Task   State        Cycles     Timeout Sleep   Task                                               
------ ------------ ---------- ------- ------- --------------------------------------------------
1      FUTURE_WAIT  1          None    None    Monitor.monitor_task                              
2      READ_WAIT    1          None    None    Kernel._run_coro.<locals>._kernel_task            
3      TASK_JOIN    2          None    None    parent                                            
4      SEMA_ACQUIRE 1          None    None    kid                                               
5      TIME_SLEEP   1          None    985.445 friend                                            
6      TIME_SLEEP   1          None    985.445 friend                                            
7      TIME_SLEEP   1          None    985.445 friend                                            
8      TIME_SLEEP   10         None    0.45906 countdown    

probably because aexit of the context manager waits for the tasks to finish.

I am still curious on why the print line never gets printed.

[goldcode]

probably because aexit of the context manager waits for the tasks to finish. yes it would wait for 1000 secs for all of the friend tasks and that is when parent calls await kid_task.cancel(). At any await point, you must reckon with a curio.CancelledError exception. here the exception gets thrown and the try/except block of kid() would never get executed.

[dabeaz]

Yes, this is being caused by the TaskGroup waiting for all children to exit. A core behavior of task groups is that spawned tasks are not allowed to outlive the lifetime of the task group.

I actually am a bit curious about the behavior though. If the kid() task is being cancelled without the print taking place, it means that the cancellation must be originating in the __aexit__() method of the context manager. I suppose that's fine, but raising exceptions in the exit method of a context manager is frankly a little weird. A few other parts of Curio actively try to avoid this. For example, consider this example with cancellation control:

from curio import *

async def kid():
    async with disable_cancellation():
        await sleep(5)

    try:
        await sleep(10)
    except CancelledError:
        print("Kid cancelled")

async def parent():
    t = await spawn(kid)
    await sleep(1)
    await t.cancel()

run(parent)

This prints "Kid cancelled." Hmmm. I'd need to think about this. You could probably fix it by putting the try-except around the whole TaskGroup construct.

[dabeaz]

This change of moving the try-except would probably be a good example to add to the tutorial to illustrate the behavior of TaskGroup. That is, by moving the try except, the original kid() task merely sits around watching the friends play instead of playing at the same time.

[skgbanga]

Thanks for the replies. Here is something interesting then:

If I change the kid's function to be:

async def kid():
    print('Building the Millenium Falcon in Minecraft')

    try:
        async with curio.TaskGroup() as f:
            await f.spawn(friend, 'Max')
            await f.spawn(friend, 'Lillian')
            await f.spawn(friend, 'Thomas')

            await curio.sleep(1000)

    except curio.CancelledError:
        print('Fine. Saving my work.')
        raise

The output is

....        
We're leaving!       
I warned you!        
Fine. Saving my work.                      
Thomas going home    
Lillian going home   
Max going home       
Leaving!             

Which means that if the TaskGroup has some tasks which have some cleanup work to do ({name} going home in this case), that can actually happen outside the async with curio.TaskGroup. (Because Fine. Saving my work was printed before those)

I am also curious about the example you gave:

from curio import *

async def kid():
    async with disable_cancellation():
        await sleep(5)

    try:
        await sleep(10)
    except CancelledError:
        print("Kid cancelled")

async def parent():
    t = await spawn(kid)
    await sleep(1)
    await t.cancel()

run(parent)

Since kid cancelled gets printed, it means that CancelledError was eventually raised when kid did an await call. e.g. if I change the kid's function to:

async def kid():
    async with disable_cancellation():
        await sleep(5)

    print('Doing some expensive work')
    try:
        await sleep(10)
    except CancelledError:
        print("Kid cancelled")

Then Doing some expensive work will get printed. I guess this is perhaps because the control flow has to transfer back to the loop before any action is taken, but it does feel a bit weird. I think it is more natural for CancelledError to be thrown at the end of context manager in this case. (and thus letting all the things within the context manager to be executed without cancellation, but no more than that)

[dabeaz]

Ooh. Interesting. I think this is related to something I had noticed while debugging something else last week---namely, the use of a non-blocking cancellation in Task Groups seemed to be weird. I thought I had eliminated all uses of that, but I see that I didn't. I'll push a fix that should address this.

[skgbanga]

I can confirm that with the fix, TaskGroups tasks' finally block is executed before the parent.

Please feel free to close this ticket, and thanks for answering the questions.