General docs update -- braindump to be written up

[joshring]

• Async code has its problems, we are planning to implement more flexible concept which should generalise well across not just IO, but work well for task queues, embedded realtime programming and distributed systems.
• C3 can work with C++ if C++ exposes a C-compatible interface to it's code hiding the C++ specific symantics.

Async Design Problems

• Async code solves the problem of single threaded code wanting to execute non-blocking tasks concurrently and was popularised by Javascript.
• This design has lead to concepts like function colour, where async functions only works with other async functions and this is common in Javascript and Python where it has caused extra complexity and has split their ecosystems.
• This design does not work well for other use-cases, like multithreaded compute or hard realtime applications.

Coroutines

• The approach used by Go in contrast is to create cheap coroutines (aka "goroutines" or "green threads") and allow code running on those coroutines which would wait on IO to give the thread to another piece of work which needs to make progress while it waits on IO.
• This design works well for IO, but does not work nearly as well for other use-cases, like multithreaded compute or hard realtime applications.

The General Solution Proposal

• Flexible syntax to allow you to make the trade-offs appropriate to your use-case.
• Customise your own scheduler, with options suitable to your use-case.
• Control dependencies between tasks
• Control timeouts
• Control what can be cancelled and when
• Control task priority
• Control interrupts to prioritise tasks
• Define actions on failure, eg logging, restart task etc.
• This is at proposal stage and needs your help, more details of the proposal.

TODO:

• We should add “how to iterate over a map” to the FAQ
• Probably also something why interfaces are limited and how they should be used

[lerno]

I would say that there is no general solution, and trying to find general solutions are doomed to failure.

What we should / can do is to provide different library features, with varying degrees of control and behaviour. For example, we might provide different kinds of thread pools, event handlers/listeners etc. These should come from proven solutions.

EVERYONE wants to solve concurrency, but what people tend to forget is that there will be no on-size-fits-all solution.

Let's instead focus on providing good niche solutions and good libraries to support those solutions and others that people might want to create.

[joshring]

flexible syntax to allow you to make the trade-offs appropriate to your use-case. Customise your own scheduler, with options suitable to your use-case.

I think we agree, let's find some suitable schedulers we can borrow from