Question: Will it be possible to define an Async/Concurrency effect?

[ProofOfKeags]

I'm currently doing research into understanding why it is that certain effects are absent from various effect systems. Of particular importance to me is to be able to use asynchronous operations in my effectful code as many data sources can be used to fetch things concurrently, greatly improving performance. It appears that fused-effects is unable to support this in its current design. Since eff takes a fundamentally different approach with the GHC Primops proposal (great proposal btw), will this re-enable the ability to define asynchronous operations?

Related: are coroutines and asynchronous operations the same thing? I ask because coroutines are explicitly mentioned in the Delimited Continuations proposal as one of the motivating factors. To my uneducated self these seem like the same thing, but it is also tough to tell.

[lexi-lambda]

It depends on what you want your hypothetical Async effect to be able to do.

If you just want to be able to run some IO actions concurrently, there’s no problem. You can just write, say,

import qualified Control.Concurrent.Async as Async

parallelFetch :: Member IOE effs => Eff effs Blah
parallelFetch = do
  (result1, result2) <- liftIO $ Async.concurrently fetch1 fetch2
  ...

and that works just fine.

But trouble arises when you want your fetch1 and fetch2 actions to be able to perform non-IO effects—that is, you want them to run in the enclosing Eff monad. For that, you want an operation with a type like this:

concurrently :: Member Async effs => Eff effs a -> Eff effs b -> Eff effs (a, b)

Such an operation would obviously be valuable, but it raises all sorts of hairy questions:

1. If both arguments to concurrently modify state—provided, for example, by a State handler—how is the state synchronized between them?

2. If one of the arguments performs an abort—for example, to an Error handler—what happens to the other thread of execution?

3. Worst of all, if one of the arguments suspends execution (i.e. it captures the continuation)—yielding, for example, to a Coroutine handler—do all threads of execution get suspended and later resumed? (And if so, how is this implemented?)

While all these questions have potential answers, it’s unclear what precisely those answers should be. To my knowledge, there is no existing formal model of algebraic effects that handles concurrent threads of execution, and that is not simply an oversight; I genuinely don’t know what the right semantics is.

So until that changes, the answer is pretty simple: you can do whatever concurrency you want via IO, and you can spawn Eff subcomputations inside your concurrent IO actions if you’d like, but eff will not provide any automatic mechanism to allow effectful operations to cooperate across that boundary. You have to manage that yourself.

[avanov]

While all these questions have potential answers, it’s unclear what precisely those answers should be. To my knowledge, there is no existing formal model of algebraic effects that handles concurrent threads of execution, and that is not simply an oversight; I genuinely don’t know what the right semantics is.

There's "Structured Asynchrony with Algebraic Effects" paper, the 3.x sections, as far as I understand them, cover the bullet points mentioned above.

[arybczak]

FWIW I believe effectful solves this problem.

1. There's an AsyncE effect as a thin wrapper over the async library that allows you to run Eff computations asynchronously.
2. The Error effect uses exceptions underneath, so error propagation from Async actions to the parent thread (and cleanup) "just works".
3. There are two flavors/interpretations of State and Writer effects - thread local (represented as a pure value internally) and shared (represented as an MVar internally) which do what you'd expect them to.