Question about a pipeline design pattern

[olleharstedt]

Hi!

I'm looking for a pipeline-based design pattern where each side-effect is run through a yield. Something like

function doSomething()
{
  if ($whatever) {
    yield $io->db->query('UPDATE bla ...');
  } else {
    yield $io->file->write($filename, $data);
  }
  // More logic...
}

(Please disregard undefined variables.)

The point would be that all side-effects that do not depend on the result of the side-effect can be executed concurrently. So IO concurrency would be the default behaviour. Is this something that can be achieved with Amphp?

[kelunik]

Hey @olleharstedt, yes, that's definitely possible. The implementation will depend on the requirements, obviously. For a smaller pipeline you might be able to analyze the dependencies manually and write the code down, but for a larger or dynamic set of tasks, it'll be easier to define a framework around the tasks that handles priorization / ordering. https://github.com/dbalabka/dag might be a good starting point.

[olleharstedt]

Thank you for your feedback!

Hm, ordering should be FIFO, as expected by the flow of the code. So a graph of priority or similar wouldn't be needed (as of what I plan right now). What's needed is that the controlling code runs each side-effect and returns to logic before it's finished. Kind of like this line:

list($result1, $result2) = yield $promises; // Both queries execute simultaneously. Wait for both to finish

https://github.com/amphp/mysql/blob/master/examples/4-multi-rows.php#L23

But yield should not wait.

Basically, this:

async(something);
async(something);
await;

should be the same as

yield something;
yield something;
return