Out of order dependencies lead to exception

[mebenstein]

Tutorial 1 mentions:

Tasks can be spawned out of order. A tasks dependencies do not need to have been before being listed through the TaskSpace.

When I tried the following:

from parla import Parla
from parla.tasks import spawn, TaskSpace
from parla.cpu import cpu

if __name__ == "__main__":
    print("start")
    with Parla():
        @spawn(placement=cpu)
        async def main_task():
            t = TaskSpace("tasks")

            @spawn(t[1], dependencies=[t[0]],placement=cpu)
            def b():
                print("b")

            @spawn(t[0],placement=cpu)
            def a():
                print("a")

            await t

I get:

ValueError: ('dependent task %s should have been spawned', 'TaskID(tasks_1, task=None)')

Which is caused by the following assertion in task_runtime.py: if dependent.task is None

It seems that this assertion makes no sense since ComputeTask is only invoked after all the unspawned dependencies are supposed to be added. Thus this assertion can never be true, maybe it should be removed?

[wlruys]

I agree, this seems to have been added and not caught in https://github.com/ut-parla/Parla.py/pull/112. Thanks for raising this!

I'll remove the check and run through the tests to make sure everything is okay.

Also as a heads up, we've been rewriting our runtime from scratch in C++ for Parla 0.3 where we've made these mechanisms a bit more robust than storing waiting tasks in a global dictionary :) We have a few more features to refine before release but it will hopefully be within the next month.

[wlruys]

Seems like there's a few mutex conflicts for this case since it hasn't been tested in a while. We're seeing if we can resolve it.