consolidation / robo

Modern task runner for PHP
http://robo.li
Other
2.66k stars 305 forks source link

Question about parallel execution of multiple collections #1000

Open t-heuser opened 3 years ago

t-heuser commented 3 years ago

Hey everyone,

I got a question about this awesome tool but unfortunately I didn't find an answer for it, so I'll leave it here.

I want to execute multiple collections at once (parallel). That means that I have X collections with tasks. The tasks inside the collections should be executed one after another. I want these X collections of tasks to be executed parallel now. How can I achieve this? I cannot pass a collection to ParallelExec and I did not find any way to do achieve my goal.

Here is a code example to show what I mean:

class RoboFile extends Tasks
{
    function test()
    {
        $taskCollections = [];

        $collection = $this->collectionBuilder();

        foreach ($array as $item) {
            $collection->addTask(
            // do things
            );

            // continue to add tasks to current collection until condition is met to split tasks
            if (...) {
                continue;
            }

            $taskCollections[] = $collection;
            $collection = $this->collectionBuilder();
        }

        //exec all collections in $taskCollections parallel (obviously not working as passing wrong type)
        $this->taskParallelExec($taskCollections)->run();
    }
}
greg-1-anderson commented 3 years ago

Robo doesn't support this. You could use taskParallelExec to run multiple copies of Robo, and bundle each of your task collections as individual commands, though.

greg-1-anderson commented 3 years ago

You could also try using php threads, and put one collection in each thread.

t-heuser commented 3 years ago

@greg-1-anderson Thanks for the tips. I was able to implement my desired functionality. Now another question arised. How can I get the output of a failed task executed in parallel? There seems to be no way to get the message from the result returned by the run() method. It just outputs the executed command which failed and says that it exited with code 1. But I did not find any way to get the message from the command itself. How can I do this?

$result = $parallelExec->run();
 if (!$result->wasSuccessful()) {
     // outputs command and its exit code but not the output of the executed task
    $this->say($result->getMessage());
}