All the tutorials: we need motivating (advanced) examples in our docs

[goodboy]

Clearly, we need some fancier and more immediately practical examples in our readme (says every person who's glanced ever), so I figured I'd start list of what I think would be coolio.

---

A "worker pool" example copying concurrent.futures's prime number calculator

• [x] it is the example for the ProcessPoolExecutor (ugh) so we'll need to probably mock up the executor api and show the SC way of doing things

---

The multiprocessing.Pool examples:

• [ ] the pool of workers example
• the ship tasks to workers over queues example (found at the bottom)

---

A "controlled fork blast" example

• [ ] this would be an example of creating a fork bomb out of nested nurseries which can be "controlled" via a SIGINT from the user once they've decided that enough CPU has been crushed; this demonstrates very succinctly the true power of SC.
• we might want to monitor system resources for this test to avoid making it impossible to send input whatsoever

---

A real-time streaming sorting algorithm

• [ ] I was thinking of implementing an async-ified version of pythonmaxmin which is based on the paper here
• Another thing we might be able to show off is performing this sort on multiple streams in parallel and using a trio-ized version of an Rx merge to serialize results.

---

Target tutorial set:

this is the back of hand list I can think of:

• basic spawning and actor lifetime management
• nested actor trees and inter-actor rpc
• basic streaming
• advanced 2-way streaming
• debugger usage
• daemon structures and (micro)serivice design
• fancy recipes and tractor.extras

---

Lurker opinions always welcome.

We might actually need to pick the best of these and then stick the rest in a parallelism tutorial. Actually, we might want a streaming, shared state, and parallelism tutorial in the new docs.

[parity3]

Wow, some of these, particularly the stream sorting may do well as their own separate repos/libraries!

Another possible example is to implement one of the function-as-a-service protocols with some tuning goodies.

[goodboy]

Another possible example is to implement one of the function-as-a-service protocols with some tuning goodies.

@parity3 sweet idea. I'd welcome a PR for something that too 😉

function-as-a-service

Also check #182, I think the caps based RPC is going to be here to stay the more I've thought about it. I actually have been dreaming up auto-generating capn proto schema from python's type annotations.

[parity3]

Another piece of code that may be nice to see is for a tractor controller app to run as a daemon that some other sync or polyglot app could call into, and be used as a glorified task queue implementation. You could make it compatible with celery or dramatiq or whatever.

[goodboy]

You could make it compatible with celery

I was gonna say, IMO if you pair tractor with nanomessage or zeromq you can pretty much implement celery somewhat trivially, but it'll be rabbitmq free / tractor broker kinda thing.

That might be good further down the line to demonstrate how with the right primitives from the outset these specialized message pattern designs like celery (which AMQP is just a special case of something you can build with brokerless based scalability protocols and/or an actor model) are super easy to create without a giant slew of components / configuration.

This was kind of always the intention from the outset: make the zeromq of micro-service systems showing how most message oriented designs are just special cases if you start with the right primitives. This will be extended by theory around security principles from capability based sec and how you can build ultra-resilient supervisor trees with this approach.

[goodboy]

Hmm probably worth linking celery/ceps#27

[goodboy]

@parity3 yeah maybe a "micro-service design" tutorial is in order as well.

I think we'll end up picking the simplest / hippest of these examples and toss the rest in a "parallelism tutorial" in the docs.