tj / co-parallel

Execute thunks in parallel with concurrency support
109 stars 9 forks source link

Document why this is better than yield array #8

Open fzaninotto opened 9 years ago

fzaninotto commented 9 years ago

Why is co-parallel better than simply yielding an array within co? I can't find any documentation about that, and my tests don't show any particular advantage:

// tasks is an array of 200 async tasks simulated by setTimeout, cf #3
var start = Date.now();
yield parallel(tasks, 10);
console.log('total used %dms', Date.now() - start);

This takes 170ms on average, while:

// tasks is an array of 200 async tasks simulated by setTimeout, cf #3
var start = Date.now();
yield tasks;
console.log('total used %dms', Date.now() - start);

This takes 117ms on average

juliangruber commented 9 years ago

by limiting the maximum concurrency you can control both cpu and memory usage of the task

fzaninotto commented 9 years ago

OK, so parallel is supposed to be slower than yield array by design, is that so?

juliangruber commented 9 years ago

co-parallel will be faster when the array you yield is too big so the node proc would reach its limits, and it will help to ensure not too many sockets are open at once etc.

kilianc commented 9 years ago

what I don't like about this is that we have N * Array.length functions to allocate instead of having a better co-each, am I correct?

kilianc commented 9 years ago

just to be more precise: yield* parallel(list, action, max)