Is it possible to write combined `wait for` tasks?

[bcho]

Just like the .all in Q and .when in jQuery dfd

Maybe it will look like this in Kal:

# taskA and taskB can be run asynchronously
# and using the combination will make the code logic more clear
wait for x from taskA(foo) and y from taskB(bar)

print x, y

Thanks :)

[rzimmerman]

Maybe something like:

in parallel
  wait for x from taskA foo
  wait for y from taskB bar

print x,y

would be useful?

[bcho]

@rzimmerman

looks good to me, but it just looks like created another block, would it be confusing?

[Pomax]

to keep things readable, "in parallel" is not as good as "and". Keeping it English, what about

independently
    wait for x from taskA foo
    wait for y from taskB bar
print x, y

[bcho]

@Pomax This is the same problem about wait for. IMO, using ascii symbol should be better.

[rzimmerman]

@bcho, a lot of people share your sentiment that they prefer special characters, mainly for readability. I've decided instead to focus on making the language easy to pick up and softening the learning curve. As a result Kal relies on syntax highlighting to make things readable. Obviously, this is an issue on github right now since pygments doesn't support Kal (yet).

I can't think of a better way to do this than in a block. Something like @Pomax's original syntax looks nice but breaks down if you want to run 4-5 things in parallel. Any ideas?

@Pomax, I agree that in parallel is kind of ugly. I think independently might also be confusing, though. What about making a wait for block?

wait for
  x from taskA foo
  y from taskB foo

print x, y

My only concern would be that it isn't super explicit that those tasks run in parallel. It's still a pretty simple concept to explain, though.

[Pomax]

Hmm, what about making it slightly more explicit, so that the syntax shows it's parallel/independent/etc actions:

wait for x, y
    x from taskA foo
    y from taskB bar

print x, y

that way there's clarity that we're waiting for two things at the same time, and then the block tells us where those values are supposed to come from.

[bcho]

@Pomax 's wait for x, y looks much better, but there are also some different contexts I am concerning:

• safe wait for

What if taskA may use safe wait for while taskB needn't

• Non-return tasks?

Maybe using this kind of syntax?

wait for x, 
    x from taskA foo
    taskB bar

But I think the x, may look confusing.

• Is it possible to create a parallel wait for block from a list of tasks?

For example:

tasks = [taskA, taskB, taskC, taskD, ...]

wait for returns
    from tasks

Maybe using a list of tasks can just mean they're run parallel?

But I don't know how to specific the arguments for each task. Maybe use currying to wrap the task and arguments together?

[rzimmerman]

Well right now (with the current build) you can do:

tasks = [[taskA,[1,2,3]
             [taskB,[0]]
             [taskC,[]]]
returns = []
for parallel taskInfo in tasks
  wait for x from taskInfo[0].apply taskInfo[0], taskInfo[1]
  returns.push x

But clearly that's not a good enough solution.

I do like what you have above, but with the following tweak:

wait for
  x from taskA(1,2)
  safe y, z from taskB()
  taskC()
print x, y, z

The only things allowed in a wait for block would be this type of clause ([safe] [ARGS from] EXPRESSION). 99% of the time it will look nice, like:

wait for
  user1 from db.getUser(1)
  user2 from db.getUser(2)
  admin from db.getAdmin()
  save changes
print 'admin is playing' if user1 is admin or user2 is admin

[bcho]

err, the list form still looks too tweaky (and too much to type when coding), can we write in wait for taskList but actually generate the code like for parallel ?

[Pomax]

Just a thought, but if tasks have no inputs, or have inputs but none of them need to be waited for in the same block, wouldn't it make sense to have the task fire in parallel by default? That way

wait for x, y
  x from taskA foo
  y from taskB bar

can be two async calls and will just do the right thing, and

wait for x, y
  x from taskA foo
  y from basedOn(x)

will have y waiting for x before being called.

[rzimmerman]

@Pomax, that is definitely more complicated but kind of cool. I think async.js does something like this. I worry about it getting too complicated for new users, but it does let you do something neat:

wait for x, y, z
  x from taskA()
  y from taskB(x)
  z from taskC()
  taskD()
print 'done'

Which would:

1. Kick off taskD without stopping or waiting for it to finish.
2. Start taskA and taskC at the same time
3. Start taskB whenever taskA finishes, regardless of what taskC is doing
4. Print done when tasksA-C are finished.

I think that's what you're getting at. It would be a lot of work, so maybe we could split this into two features:

1. The simple parallel block I suggested for now since it's straightforward to implement
2. Something very similar to the wait for block you described above some time in r0.5.x. This requires more thought and effort.

[rzimmerman]

Also, @bcho, I do have on the list (gh-44) adding wait fors to list comprehensions, so you could eventually do something like:

task get_user(id)
  ...

user_ids = [...]
users = [wait for get_user(id) for parallel id in user_ids]

Which would (theoretically) spawn parallel tasks to get_user for each ID, then assemble the array, kind of like async.map.

[rzimmerman]

I added the run in parallel structure to master (not on npm yet). I'd like to keep the discussion open, but push the more complicated wait for block to a future release.

[bcho]

@rzimmerman how about this form?

run in parallel
  x from taskA foo
  y from basedOn(x)

I didn't find it in the test case.

[rzimmerman]

@bcho that should be working. This section from tests/parallel_block.kal has one (with multiple return values):

run in parallel
  c1()
  wait for y, yy from c2()
  z, zz from c3()
  safe wait for c1()

[Pomax]

As a stale issue, can this be closed? It's impossible to remove from github.com/issues/mentioned and the issue itself clearly hasn't had anyone run into this problem for almost four years.