Closed LukaJCB closed 6 years ago
There can be no principled, leak-free, monadic concurrency in functional programming without MonadBracket
and MonadFork
, or equivalents, including interruption semantics analogous to Scalaz 8 / Haskell / PureScript.
Interruption is fundamental to composability, and must be baked into the lowest layer of the stack, which is the effect monad that drives the application.
Not only is such a thing possible to do in a lawful fashion with precise semantics, but it has been done, in Scalaz 8 IO and elsewhere. The fact that existing libraries do not support these semantics is irrelevant because (a) existing libraries can always be improved, or alternatively (b) type class laws can be weakened so as to permit "no op" implementations.
I happily donate the following type classes to the project:
trait Forked[F[_], A] {
def interrupt(t: Throwable): F[Unit]
def join: F[A]
}
trait MonadBracket[F[_]] extends MonadError[F, Throwable] {
def bracket[A, B](acquire: F[A])(use: A => F[B])(release: (A, Either[Throwable, B]) => F[Unit]): F[B]
def never[A]: F[A]
}
trait MonadFork[F[_]] extends MonadBracket[F] {
def fork[A](fa: F[A]): F[Forked[F, A]]
def raceWith[A, B, C](l: F[A], r: F[B])(
finish: Either[(A, Forked[F, B]), (B, Forked[F, A])] => F[C]): F[C]
}
These are very small and flexible type classes, while providing just enough power to construct correct, composable, and leak-free software. All methods have low-cost implementations which may not have the full capabilities of more extensive implementations but which can lessen author burden.
MonadBracket
must be a super class of Sync
. That is to say, it does not make any sense to have a Sync
without the ability to bracket
(bracket
gives meaning to the notion of monadic operations on foreign effectful code). Separately, I'd also argue that Async
and Sync
should be unified because there is nothing intrinsically useful which is a Sync
and not Async
.
MonadFork
is necessary for safe, leak-free concurrency. That is, any F[_]
which does not have a MonadFork
should not be used for concurrency. In no case should concurrency be implemented on top of an F[_]
that does not support MonadFork
because it will be broken by construction.
Of course, a concurrent F[_]
could support more than just MonadFork
, but MonadFork
provides the bare essentials necessary to implement higher-level, composable, safe combinators on top (parMap2
, concurrently
, etc.).
Concurrent libraries like FS2 and http4s must be able to rely on existence of MonadFork
. Simpler libraries that do not have concurrent needs do not have to use MonadFork
and will therefore benefit from much simpler IO implementations.
@alexandru
Scalaz 8 IO fully linearizes interruption / finalization. Finalization will never occur out of order or concurrently, but rather, it will be done in the correct order and fully sequentially, post-successful interruption, after user-defined effects have returned control to the runtime. This ensures implementation details are not leaked (the logical model of a linear fiber is maintained) and provides a simple reasoning model that makes it easy to write correct code.
@jdegoes I feel that we aren't communicating well, I don't understand why, maybe we are not using the same language. I have some gaps in my education, I'm actually trying to finish college right now (12 years later); but to me "the logical model of a linear fiber is maintained" sounds like technobabble.
You have to give me some credit though, because the Monix Task
was born 2 years ago and it has a very similar cancellation and evaluation model, so if we are to collaborate, which we should because we can do awesome things apparently, we need to pay more attention to each other 😀
Scalaz 8 IO fully linearizes interruption / finalization. Finalization will never occur out of order or concurrently, but rather, it will be done in the correct order and fully sequentially, post-successful interruption
You can drive several trucks through that statement, because it's carefully worded to ignore the elephant in the room that I mentioned in my samples above.
provides a simple reasoning model that makes it easy to write correct code.
Not true:
bracket
and cancellation, which couldn't happen without cancellation — https://gist.github.com/alexandru/f30b0c8b3920e7d8a8a6ecf018c0aaecThat Scalaz 8 IO code is actually behaving more or less like I expected, since I've lost nights over this for some time now — I did not even have to run your code to see it, because it's all in its signatures, but there:
Started!
Thrown! java.io.IOException: Stream closed
(run-main-0) java.lang.RuntimeException: Boo
java.lang.RuntimeException: Boo
at scalaz.effect.Sample$.$anonfun$run$5(Playground.scala:15)
at scalaz.effect.RTS$.nextInstr(RTS.scala:143)
at scalaz.effect.RTS$FiberContext.evaluate(RTS.scala:417)
at scalaz.effect.RTS$FiberContext.continueWithValue$1(RTS.scala:690)
at scalaz.effect.RTS$FiberContext.resumeEvaluate(RTS.scala:696)
at scalaz.effect.RTS$FiberContext.resumeAsync(RTS.scala:729)
at scalaz.effect.RTS$FiberContext.$anonfun$evaluate$4(RTS.scala:496)
at scalaz.effect.RTS$FiberContext.$anonfun$evaluate$4$adapted(RTS.scala:496)
at scalaz.effect.RTS$FiberContext.$anonfun$evaluate$19(RTS.scala:603)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
at scalaz.effect.RTS$$anon$1.run(RTS.scala:95)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Can you see the IOException: Stream closed
?
That's data corruption right there and no, it's not simple, it's not intuitive, I can argue backed by the actual experience of having users to support that this behavior right here is precisely what users do not expect 😉
At the same time, this is more or less the best we can do (minus some design decisions of yours that I don't like), but we need to call a spade a spade.
MonadBracket must be a super class of Sync. That is to say, it does not make any sense to have a Sync without the ability to bracket
I agree.
This is nice, but it leaks your implementation details, for which I have reasons to disagree:
trait Forked[F[_], A] {
def interrupt(t: Throwable): F[Unit]
def join: F[A]
}
Here's Monix's Task
as of 3.0.0-M3
:
def cancel[A](fa: Task[A]): Task[Unit]
// Yes, this is our join
def flatten[A](fa: Task[Task[A]]): Task[A]
Some problems:
Forked
interface is OOP and would need to be inherited, being incompatible with the type classes that we are trying to promote and this is relevant because in usage this leads to loss of fidelity in the returned types; and if we introduce it as a parameter, it's not feasible to pass it around in addition to IO
Throwable
to kill a task, for reasons that I can't get into right now — it's enough to say that I believe a cancelled task should be non-terminatingFor bracket
this is insufficient:
trait MonadBracket[F[_]] extends MonadError[F, Throwable] {
def bracket[A, B](acquire: F[A])(use: A => F[B])(release: (A, Either[Throwable, B]) => F[Unit]): F[B]
}
I already explained above why, we need to make a difference between interruption and normal finalization and even your own code confirms it.
For MonadKill
this also leaks your implementation details:
trait MonadFork[F[_]] extends MonadBracket[F] {
def fork[A](fa: F[A]): F[Forked[F, A]]
def raceWith[A, B, C](l: F[A], r: F[B])(
finish: Either[(A, Forked[F, B]), (B, Forked[F, A])] => F[C]): F[C]
}
Compare with Monix's Task
as of 3.0.0-M3
:
def start[A](fa: Task[A]): Task[Task[A]]
def racePair[A, B](fa: Task[A], fb: Task[B]): Either[(A, Task[B]), (Task[A], B)]
I raised this issue on the gitter channel and got a lot of positive feedback. We removed
IO#ensuring
some time ago and realizedMonadError
is not powerful enough to implement it (See here). I'd like to propose something in the vein ofMonadBracket
described in this article. It might also help with providing a resource safeParallel
experience.Relevant gitter discussion: