search

cujojs / when

A solid, fast Promises/A+ and when() implementation, plus other async goodies.
Other
3.44k stars 396 forks source link

Introducing when/distribute operation for task execution. #375

Closed jorgemsrs closed 9 years ago

jorgemsrs commented 9 years ago

when/distribute is a new operation for task execution. It's much alike pipeline but with 2 fundamental differences:

  1. The tasks return values are cumulatively passed as inputs to subsequent tasks.
  2. The resolved promise resolves to an array with the return value of each task.
briancavalier commented 9 years ago

Hey @jorgemsrs, this is an interesting variant. I'll take a closer look at the code a bit later today. In the meantime, I'd like to get a feel for when someone would use it. Could you describe the use case(s) where you're using it? Thanks!

jorgemsrs commented 9 years ago

Hi @briancavalier. This came from the generalisation of one use case I had where I needed the resolved values of more than one of the previous tasks in subsequent ones. This is then a design to help avoid global variables when similar cases occur.

Imagine the following: task1: open file and return content task2: grep expression task3: grep expression2 (needs task1 result) task4: combine results (needs task2 and task3 results) ...

This is in effect a task result join operation. An interesting detail is the final resolved value being similar to all where one can spread the pipeline operation and work the individual task results. This can be interesting for instrumentation and/or logging purposes.

I reckon this is not a behaviour commonly needed yet I decided to share the code and submit it to the community opinion anyway.

cheers

briancavalier commented 9 years ago

Thanks for the info @jorgemsrs.

I have some concerns about building up an argument list as distribute runs. That seems to tie each task to its position in the array. That could lead to refactoring headaches or hazards, since inserting a new function into or removing an existing function from the array would change the number of parameters that every function at a higher array index would receive. Changing the order of tasks in the array would have similar repercussions.

For example, it looks like task2 and task3 are independent (ie, they could run in parallel), but require the same input (the content). Let's say you switched the order of task2 and task3. Would that require you to change the function signature of both tasks?

Also, I think there may be simpler ways to combine tasks when you need to do this kind of thing (using previous results, etc.). Here's a quick take on the double-grep-then-combine example. Any of the "tasks" can return a promise, and putting them together only required a couple lines of code using when.try.

Any thoughts on all of that?

jorgemsrs commented 9 years ago

The proposed idiom is flawed. Tying execution order with implementation is definitely a no go.

Even the detail of having the promise chain resolve to an array of individual task executions is easily fulfilled using, for instance, join.

Thanks!

briancavalier commented 9 years ago

@jorgemsrs No problem, thanks for the discussion. BTW, even though we ended up not merging this, I appreciate that it was such a complete pull request: code, docs, and tests. Cheers!