Run tasks in parrallel

[hoanguyen311]

Hello Kir, As I tried, we can not do something like this:

function test() {
    return start(
       [ loadNames('./names.txt'), loadNames('./names2.txt')] , //They should run in parrallel
        hello('Hello'),
        logName()
    );
}

It's nice if you can make it possible.

[deepsweet]

Hey!

Do you have a real example when it's really needed? Right now you can use it this way:

export function test() {
    return start(
        Promise.all([
            start(loadNames('./names.txt')),
            start(loadNames('./names2.txt'))
        ]),
        hello('Hello'),
        logName()
    )
}

I'll take a closer look at your PR soon, thanks for that.

[hoanguyen311]

It's not really necessary when I saw your solution. I think the only benefit right now is It make the code cleaner and more simple

Thanks.

[eisisig]

Forwarding arrays to Promise.all might be a good quick solution

[deepsweet]

the only thing that stops me from implementing this is an issue with loggers. I mean it will be a mess of messages from different tasks w/o any understandable order...

[deepsweet]

so I'm gonna try something like this as a first step:

before:

→ files: start
→ files: lib/index.js
→ files: start
→ files: done

after:

→ build.files: start
→ build.files: lib/index.js
→ dev.files: start
→ test.files: done

[tunnckoCore]

That's cool idea and I think each-promise can come in great help here :)

So then start will have parallel, serial and concurrency control. :) Just instead of tasks.reduce use eachPromise.each(tasks, options) where options will come from first function call (were currently is reporter = console.log).

[deepsweet]

well, I finally have something. there will be a special tasks wrapper (soon) called concurrent, something like this:

export const test = () => start(
  concurrent(
    task1(),
    task2(),
  ),
  task3(),
  task4()
}

with a simple Promise.all under the good. the whole wrapper idea is to hide an implementation details. logger's stuff may be a little messed, because we can't just pass tasks runner name inside (at least because of nested runners), but I think it's not so big deal.

parallel is not the same, and we can't just use child_process.spawn() to get a real threads because we don't have a separated files. it would be really nice if we can use something like Web Workers + Blob to run "inline" function (even with requires) as a new process thread.

[tunnckoCore]

Yea, it isn't. But in scenario where we have one file in one thread, we still have serial and parallel flows.

each-promise helps me a lot and I'm going to use it in upcoming test runner, where some want to run all tests in one file without preserving defined order (hence, parallel), some want to run them in order (because hooks such as beforeEach to work). In context of "tasks" the case is absolutely the same.

each-promise is just kind of async for promises.

the whole wrapper idea

yea, wrapper looks good way to go

[deepsweet]

it's better late than never ✨ https://github.com/start-runner/concurrent

[deepsweet]

got this working with true parallel mode through child_process.spawn():

export const build = (/* my, super, args */) => start(
  env('NODE_ENV', 'production'),
  parallel(
    makeDist,
    makeES,
    makeLib
  )
);

also:

• logs for each tasks runner are collected and printed as a batch when runner finished, so no mess in console
• tasks runner arguments like start-runner build my super args are supported as well and will be passed to each runner

coming soon. weee.

[deepsweet]

https://github.com/start-runner/parallel