charmplusplus / charm4py

Parallel Programming with Python and Charm++
https://charm4py.readthedocs.io
Apache License 2.0
290 stars 21 forks source link

Enable Futures to Work Across Greenlets #195

Closed ZwFink closed 3 years ago

ZwFink commented 3 years ago

Charm4Py uses Greenlets, lightweight thread-like objects to manage suspension of execution within a chare. Currently, futures in Charm4Py can only unsuspend correctly when future creation and waiting happens in the same coroutine. This means that in the master branch of Charm4Py this code snippet will hang when trying to retrieve the value from the futures:

from charm4py import charm, Chare, Future, coro
import time

class TestChare(Chare):
    @coro
    def __init__(self):
        self.future1, self.future2 = Future(), Future()

    @coro
    def waitf(self):
        x = self.future1.get()
        print(f"Wait 1: {x}, {time.time()}")

        x = self.future2.get()
        print(f"Wait 2: {x}, {time.time()}")

    @coro
    def refresh_futures(self):
        self.future1, self.future2 = Future(), Future()

    def get_futures(self):
        return (self.future1, self.future2)

def main(args):
    test_chare = Chare(TestChare, onPE=1)
    wait1, wait2 = test_chare.get_futures(ret=True).get()

    test_chare.waitf()
    wait1.send(1)
    wait2.send(2)

charm.start(main)

This branches fixes this bug by allowing Futures to update their greenlet reference to the current execution context before yielding.