There is a common pattern (not only in the worker, but across all our components) which is difficult to describe with safe Rust:
multiple subtasks are running concurrently forever
if any of them completes (e.g. panics), all the rest should also be terminated
the termination has to be graceful — set the cancellation token and wait for the task to end
subtasks may want to split themselves into other subtasks.
Basically, it's what is called "structured concurrency"
The main complication is that to run the subtasks truly in parallel you have to use tokio::spawn which accepts a 'static future (scoped tasks have been removed), making you pass all the shared objects wrapped in Arc.
The task_scope crate looks like a possible solution, but it's very unpopular, so it should be carefully reviewed first.
There is a common pattern (not only in the worker, but across all our components) which is difficult to describe with safe Rust:
Basically, it's what is called "structured concurrency" The main complication is that to run the subtasks truly in parallel you have to use
tokio::spawnwhich accepts a'staticfuture (scoped tasks have been removed), making you pass all the shared objects wrapped inArc.The
task_scopecrate looks like a possible solution, but it's very unpopular, so it should be carefully reviewed first.