Worker pool API?

[goodboy]

@parity3 made a gitter request for a system to delegate work to an actor/task cluster or pool.

for my use case, I've got multiple "clients" that wish to use the same set of workers, coming in at sporadic times. So they'd need to connect to a running arbiter, and the arbiter needs to make sure that some workers are free before starting on the client-specified set of task chunks (as a method of resource manageement). It'd be nice to see an example for that use case, or more built in api support for such case.

And with a little follow up requirement:

also, it's not clear to me how best to implement a pattern where actors are all notified that there are no more unclaimed tasks of a set (for the actors that were involved with the task set). It would be nice for such actors to be notified so they can "wrap up" any transient data associated with the tasks they performed, aggregate / summarize it, and send it back to the client

That last part will probably be to do with a lack of docs on how actor cancellation can be done using one of ActorNursery.cancel(), Portal.cancel_actor() or a plain old trio.CancelScope.

This worker pool is something I've though about a bit and it reminded me of a couple projects:

• the routers and pools in akka
• the monitors in pulsar
• worker pools in erlang OTP for which there's a nice wrapped implementation, poolboy
• an interesting post on the need for a worker pool in rabbitmq (introduced some time ago: 2010)

More discussion and brainstorming is greatly welcome!

[goodboy]

A much delayed follow up: we'll probably start with an example in the README / docs (#163) and add a more extensive tutorial surrounding this later on.

In terms of an API being incorporated into tractor itself I'd like to see a slew of implementations before getting to set on anything.