New Project: Typelevel Schrodinger

[alexandru]

I have a proposal for a new project, called Typelevel Schrodinger.

What this project aims to do is to provide a common interface between various Task and IO like data-types and would basically be what Reactive Streams is for streaming, the purpose being to allow interoperability between various libraries.

I started an initial draft here, but would be cool if we moved this to the Typelevel organization and collaborate on it:

https://github.com/alexandru/schrodinger

Summary / Current Proposal (Updated: Apr 7, 2017 23:01)

1. we start project schrodinger-core which:
   • provides Evaluable, Deferrable, Eventual (or Effect) and Async type-classes
   • is meant as middleware, for interoperability purposes (e.g. conversions)
   • this is meant for library authors, not users
   • it should be as stable and as light as possible (i.e. no dependencies) and once we release 1.0.0 I'd like it to remain set in stone as to not create problems for the downstream
2. we start a cats-effect project which:
   • provides a reference cats.effect.IO implementation, that depends on cats-core and which also implements instances for schrodinger-core
   • this IO implementation is in fact a Task, capable of async evaluation - naming it like this would be nice because Cats already has Eval and an async IO would be a more capable equivalent of Haskell's IO and it would also leave the Task name to be used for other implementations

Open questions:

1. Is it OK to have an async cats.effect.IO? (as name, instead of Task)
2. Can project Schrodinger be moved to the Typelevel GitHub organization?
3. ... (not sure if I missed anything)

A graph of the dependencies, as I see it:

/cc @tpolecat @rossabaker @mpilquist @edmundnoble @djspiewak @non @adelbertc

(notifying here the folks that showed interest, sorry for the spam, not sure if I missed anybody)

[cquiroz]

Do you think this would be compatible with scala.js?

[alexandru]

Do you think this would be compatible with scala.js?

@cquiroz yes, it definitely is. I haven't configured that project yet for Scala.js, but it will be.

[djspiewak]

Compatibility with Scala.js definitely seems like a 100% critical requirement.

I would actually depend on Cats. We're going to need to figure out fs2 things at some later date. Part of the value of something like this, as opposed to just depending on the soon-to-come fs2-task submodule, is the fact that it would be highly integrated into the Cats ecosystem.

I would rather bring in something approximating the fs2 typeclass hierarchy (Capture, Catchable, Async, etc) rather than adding something as ad hoc as MonadDefer. This also isolates the effectful typeclasses in the effects project, which seems like logically where they belong. You already basically have this, so I'm ok with that.

I'm definitely not a fan of the ExecutionContext constraint, but I acknowledge that there's no way to apply these typeclasses to monix.Task (and similar constructs) without it. It over-constrains implementations like fs2.Task and also results in types that imply the wrong semantics (since fs2 will literally ignore the EC), but I don't see a better way.

Public Domain and/or CC0 licensing are inappropriate for software distributed in the US. Copyright law is different here in some pretty fundamental ways, notably relating to patent and copyright assignment. These are differences that "real" licenses (notably GPL and ASL) handle very carefully, but licenses like Public Domain, CC0, MIT and such essentially just ignore.

Oh, I'm also strongly averse to the Callback approach (as a separate type). I understand that it avoids the extra allocation, but it also means that it's impossible to handle error cases without extra side-effects. Additionally, the use-site syntax becomes a lot worse, since you need a full anonymous inner class. It also represents an extra type which has to be digested, resulting in higher cognitive load for users trying to learn the API.

On another random note, it's worth pointing out that the API as designed is incompatible with approaches like purescript's Aff, which is similar in spirit to what @puffnfresh and I have been experimenting with here.

[mpilquist]

Can we make the ExecutionContext parameter path dependent and let the instance decide what type to pass there?

trait Effect[F[_]] {
  type Context
  def unsafeExecuteAsyncIO[A](fa: F[A], cb: Callback[A])(implicit ctx: Context): Unit
}

[adelbertc]

I like Michaels approach more than forcing the EC

On Mar 29, 2017 09:39, "Michael Pilquist" notifications@github.com wrote:

Can we make the ExecutionContext parameter path dependent and let the instance decide what type to pass there?

trait Effect[F[_]] { type Context def unsafeExecuteAsyncIO[A](fa: F[A], cb: Callback[A])(implicit ctx: Context): Unit }

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/typelevel/general/issues/66#issuecomment-290148352, or mute the thread https://github.com/notifications/unsubscribe-auth/ABRW9J5r-1SLCzYJThn3JFmTcmHaT02cks5rqoksgaJpZM4Ms-Wh .

[alexandru]

@mpilquist

Can we make the ExecutionContext parameter path dependent and let the instance decide what type to pass there?

Unfortunately that doesn't help with abstracting over these types at the call site, where this abstraction is needed, because at the call site you have to know what that Context is, so as an abstraction Effect is no longer useful.

[mpilquist]

The call site would need an implicit f.Context, though perhaps that's too complex/unwieldy?

I really, really don't like the ExecutonContext parameter.

[djspiewak]

@alexandru

def foo[F[_], A, C](fa: F[A])(implicit F: Async.Aux[F, C], C: C) = ???

[alexandru]

I would love for that to work, unfortunately I don't see how. If the call site needs an f.Context, that needs to come from somewhere.

If we describe a generic foo that takes that F[_] : Async as parameter, you simply move the problem around. At some point, somebody needs to specify an actual f.Context instance. If we don't specify it in that foo function that @djspiewak just described, then we need to specify it at foo's call-site or further down the line.

And if the goal is to generically call unsafePerformIO, then that EC will have to be a type we can agree on. FS2 already has Strategy, which is basically the same thing (minus reportError which IMO is also useful).

I'm definitely not a fan of the ExecutionContext constraint, but I acknowledge that there's no way to apply these typeclasses to monix.Task (and similar constructs) without it. It over-constrains implementations like fs2.Task and also results in types that imply the wrong semantics (since fs2 will literally ignore the EC), but I don't see a better way.

In Monix we'll have the same problem with Task.async, for which we do not require any EC currently. So our implemented Async.create will also ignore that ExecutionContext.

That is fine though, because the Monix Task will still have the same interface as it does today, these APIs being meant for interoperability purposes only. Just to give an example, working directly with the Reactive Streams API is actually very error prone, because those aren't meant for users, but for library authors.

Also Monix is in the same boat. Our runAsync takes a Scheduler and in this case if the given ExecutionContext is not already a Scheduler then we'll have to convert, but that is fine IMO, because an ExecutionContext-like thing is all that's needed.

And I just implemented an Async[Future] instance which does use it, which is a sign that this solution is not that specific to our Tasks.

The way I see it, as soon as you're dealing with (A => Unit) => Unit, you need some sort of EC injected from somewhere, either in create or in run - and to cover both we simply require it for both 😄 and ignore it if possible.

[mpilquist]

Note that fs2.Strategy is only used by fs2.Task -- not any of the type classes. All of the FS2 combinators are implemented parametrically in F with some type class constraint. Strategy appears nowhere in these combinators.

Usage wise, who is the audience of this interop library? Do you envision libraries like http4s and doobie working parametrically with these type classes? If so, wouldn't the ExecutionContext constraint propagate all throughout the end-user APIs of these libraries?

[alexandru]

I would actually depend on Cats. We're going to need to figure out fs2 things at some later date. Part of the value of something like this, as opposed to just depending on the soon-to-come fs2-task submodule, is the fact that it would be highly integrated into the Cats ecosystem.

If everybody agrees, we can initiate a cats-effects project and then we can make use of the cats.Monad.

Public Domain and/or CC0 licensing are inappropriate for software distributed in the US.

I'm not an expert. But we can do Cats's license, that's fine.

Oh, I'm also strongly averse to the Callback approach (as a separate type). I understand that it avoids the extra allocation, but it also means that it's impossible to handle error cases without extra side-effects.

We can get rid of it, had a feeling it wouldn't fly, but I really hate that extra allocation 😄

[alexandru]

Note that fs2.Strategy is only used by fs2.Task -- not any of the type classes. All of the FS2 combinators are implemented parametrically in F with some type class constraint. Strategy appears nowhere in these combinators.

I know, but that's also because you don't interact with an "edge of the program". A library like Http4s must probably trigger that execution by itself.

Usage wise, who is the audience of this interop library? Do you envision libraries like http4s and doobie working parametrically with these type classes? If so, wouldn't the ExecutionContext constraint propagate all throughout the end-user APIs of these libraries?

Yes, unfortunately that might be a problem 😒

[djspiewak]

@alexandru What if we remove the ExecutionContext from the Async functions and simply expect that implementations which require it (e.g. Monix) capture that ExecutionContext when the typeclass instance is created? Basically the same thing that scalaz.Monad[scala.concurrent.Future] does. This is actually what fs2 does when constructing its instances, to ensure that the forked async constructor works properly.

Usage wise, who is the audience of this interop library? Do you envision libraries like http4s and doobie working parametrically with these type classes? If so, wouldn't the ExecutionContext constraint propagate all throughout the end-user APIs of these libraries?

Are there strong objections to just creating a simple implementation? It wouldn't need to be particularly elaborate, and we can punt on questions like cancelability by just shooting for either absolutely minimal or absolutely composable. For example, define type IO[A] = Free[λ[a => () => Either[Throwable, a]], A], implement sealed trait ContT[F[_], R, A] and then define type Task[A] = ContT[IO, Unit, A]. It won't be anywhere near as fast as Monix's or fs2's Task, but it's perfectly serviceable (e.g. I'm pretty sure it's faster than Scalaz's Task) and it would be able to implement Async and friends.

Alternatively, we can just copy over fs2's Task. It's not perfect for everyone's use-cases, but it's just about the simplest thing that could work (other than ContT[IO, Unit, A]). The effect typeclasses ensure that Monix (and more) can still have their own effect type, and it will still work as a first-class citizen in any environment which isn't practically forced to use a concrete effect (e.g. http4s), and all other use-cases can be handled by ensuring that we keep in mind seamless conversions.

[notxcain]

I've found it very useful to have a CaptureFuture type class, for interaction with Future base API:

trait CaptureFuture[F[_]] {
  def captureF[A](future: => Future[A]): F[A]
}

Instances should defer Future[A] creation until actual execution. So there is no instance for Future itself.

[alexandru]

@notxcain that's the same thing as the described Async.create, because you can describe your captureF in terms of it.

Async[F].create { cb =>
  future.onComplete {
    case Success(v) => cb.onSuccess(v)
    case Failure(ex) => cb.onFailure(ex)
  }
}

Note that for wrapping Future APIs, because of Monix's design, we can do one better by getting rid of the needed ExecutionContext which happens when creating any Future, since it can get injected by Task itself:

Task.deferFutureAction { implicit ec =>
  yourFuture()
}

Unfortunately we can't abstract over this.

[notxcain]

Also I think there should be no instance of Effect for Future, because unsafeExecuteAsyncIO is meaningless, as it is already being executed. However, there could be an instance of Effect[Kleisli[Future, Unit, ?]] with unsafePerformIO = kleisli.run(()).

[alexandru]

Also I think there should be no instance of Effect for Future, because unsafeExecuteAsyncIO is meaningless, as it is already being executed.

It wouldn't be meaningless, because Effect is about getting that value out of the F[_] context, and definitely not about suspension of effects.

[notxcain]

@alexandru then name of unsafeExecuteAsyncIO is a bit misleading.

[alexandru]

@djspiewak

What if we remove the ExecutionContext from the Async functions and simply expect that implementations which require it (e.g. Monix) capture that ExecutionContext when the typeclass instance is created? Basically the same thing that scalaz.Monad[scala.concurrent.Future] does. This is actually what fs2 does when constructing its instances, to ensure that the forked async constructor works properly.

I can see three problems with that:

1. Async wouldn't be a type-class anymore, because instantiation now depends on a second parameter, besides the F[_] type and my concern here is inefficiency, because if you have to create a new instance every time an Async[F] is needed, this is no longer a zero cost abstraction
2. We can no longer express the UnsafeIO OOP interface, which would be useful for hiding F[_] for certain use-cases ... this was @rossabaker's main complaint and even if he'll want to use any of this or not for Http4s (I'm pretty sure he won't), hiding F[_] is useful and OOP subtyping is the best mechanism we have for hiding it (e.g. Liskov Substitution Principle and all that)
3. If Effect.unsafeExecuteAsyncIO no longer takes an ExecutionContext, it's only logical to do the same for Async.create as well - and FS2 needs that in Async.create, so will FS2 also take an implicit Strategy for creating an Async[Task] instance, or will it use a stack-unsafe version?

I know how higher-kinded polymorphism works, I've had my own share. Basically most of the time you're deferring to the user the responsibility of executing the thing (unless you're dealing with a Comonad). This Async / Effect / UnsafeIO abstraction would be for those rare use-cases where the library needs to trigger that execution.

For example we can have routines provided by both Monix and FS2 that convert back and forth between these task types, without Monix depending on FS2 or vice-versa. Think of an operation like:

fs2.Task("sample").to[monix.eval.Task]

Conversion would also work between Task and Future and this is important when a generic F[_] is involved. For example if you have a stream-like type, you can then describe a foreach that works as Scala users expect it to:

def foreach[F[_], A](fa: F[A])(cb: A => Unit)
  (implicit A: Async[F], M: Monad[F]): Future[Unit] = ???

You cannot express such conversions without having a way to extract that value from F[_] and even though this to function would need an implicit ExecutionContext in my proposal, the alternative is to not have this function at all.

But indeed, there are differences in these APIs, an ExecutionContext works like an interpreter for async evaluation and putting it in that API makes sense for implementations that use some sort of ExecutionContext in their implementation. But implementations have different designs, priorities and it's precisely these differences in evaluation that make them special.

So at this point I don't have an opinion on how to proceed. If you think an EC-less Async class would still be useful, then that's fine, but I'm less enthusiastic about it.

[alexandru]

So I am updating the code, taking out the EC, lets see where that takes us.

[alexandru]

OK, updated the code in https://github.com/alexandru/effects/tree/master/shared/src/main/scala/org/typelevel/effects with:

1. removed ExecutionContext from the API
2. replaced Callback with Either[Throwable, A] => Unit
3. renamed functions on Effect to unsafeExtractAsync and unsafeExtractTrySync, mirroring extract from Comonad (PS: not using unsafePerformIO, run, runAsync or other variations of those is on purpose)

Let me know what you think.

[djspiewak]

Async wouldn't be a type-class anymore, because instantiation now depends on a second parameter, besides the F[_] type and my concern here is inefficiency, because if you have to create a new instance every time an Async[F] is needed, this is no longer a zero cost abstraction

I believe it can be encoded as an implicit value class, if allocations are your primary concern. Escape analysis gets 99% of this stuff in practice though, so most of the penalty is felt during JVM warmup. Finally, calls such as unsafeExtractAsync (I like the name, btw) are not generally something you're doing in the hot path, meaning that performance is far from a primary concern.

We can no longer express the UnsafeIO OOP interface, which would be useful for hiding F[] for certain use-cases ... this was @rossabaker's main complaint and even if he'll want to use any of this or not for Http4s (I'm pretty sure he won't), hiding F[] is useful and OOP subtyping is the best mechanism we have for hiding it (e.g. Liskov Substitution Principle and all that)

I haven't looked at the code in the last few hours, so maybe I missed that one. I really don't think @rossabaker using something like this is in the cards. Ross is primarily interested in efforts like this in so far as they can achieve a standard, concrete Task that he can just use. In lieu of a standard concrete Task, he's just going to default to fs2.Task. Abstracting over the effect is a non-starter for him both because of of the F[_] and because of the proliferation of implicits.

If Effect.unsafeExecuteAsyncIO no longer takes an ExecutionContext, it's only logical to do the same for Async.create as well - and FS2 needs that in Async.create, so will FS2 also take an implicit Strategy for creating an Async[Task] instance, or will it use a stack-unsafe version?

It would close over the EC. Note here and here, neither of which capture an EC. This all works because of this, which is doing precisely what I suggest: capturing the Strategy.

I know how higher-kinded polymorphism works, I've had my own share. Basically most of the time you're deferring to the user the responsibility of executing the thing (unless you're dealing with a Comonad). This Async / Effect / UnsafeIO abstraction would be for those rare use-cases where the library needs to trigger that execution.

I'm… not sure what you mean by this? It's clear that the abstractions are intended to provide the ability to both introduce and eliminate effect capture, and I think that is an excellent goal and worth abstracting over. This was never really in dispute.

For example we can have routines provided by both Monix and FS2 that convert back and forth between these task types, without Monix depending on FS2 or vice-versa.

Right, and functions like this (your to example) are precisely why this sort of thing is really great in principle.

Let's loop back to the concrete implementation suggestion though… Is there a reason not to provide a concrete "reference" implementation along with these abstractions?

[alexandru]

Finally, calls such as unsafeExtractAsync (I like the name, btw) are not generally something you're doing in the hot path, meaning that performance is far from a primary concern.

OK, you're maybe right.

Let's loop back to the concrete implementation suggestion though… Is there a reason not to provide a concrete "reference" implementation along with these abstractions?

I don't know what to think of that one.

My first instinct is to be selfish though. If there is a reference implementation, people will just use that, just like people are using Scala's Future and Scala's Either, because they are good enough.

This is an entirely subjective, superficial, wishy-washy feeling, but personally I never got into Scalaz because of its big, occupy-Scala nature, which might be totally unjustified, but in Scala's ecosystem I get the feeling we tend to prefer monoliths. But then my opinion is biased, of course, since naturally I want people to use Monix's Task.

[djspiewak]

My first instinct is to be selfish though. If there is a reference implementation, people will just use that, just like people are using Scala's Future and Scala's Either, because they are good enough.

This is an entirely subjective, superficial, wishy-washy feeling, but personally I never got into Scalaz because of its big, occupy-Scala nature, which might be totally unjustified, but in Scala's ecosystem I get the feeling we tend to prefer monoliths. But then my opinion is biased, of course, since naturally I want people to use Monix's Task.

I 100% agree this is a concern. This is part of why I was thinking that we bias the reference implementation for composability/formal-coolness. We could make it clear in the documentation that "this works, and it's convenient and cool, but if you're putting code in production you should use Monix's or fs2's Task instead". The problem it solves is really providing an "out of the box" IO type for Cats, which is something that I find myself needing far more often than you would think, as well as a type which is sufficient for http4s to assume.

Then again, http4s is already written against fs2 and unlikely to change on that point, so maybe http4s doesn't really need a "standard Task" since it already has a very direct and natural concrete type to select. So perhaps the big win here is really the ability to represent converters like your to function, rather than any reference, "batteries included" goal.

[rossabaker]

I've had a hard time selling Cats to experienced functional programmers because the IO story is so murky. I spoke to people at the conference last week who see fs2.Task and monix.eval.Task and see two nice tasks and are afraid to adopt either because "what about the other?" Library developers want people to use their library, but app developers want libraries that click together. It's so simple in Scalaz: scalaz.concurrent.Task is flawed, but it's right there, it's integrated with the core library, and it has great network effects. This choice is missing from, and holding back, Cats.

A production-grade task in a library like this might get adopted in blaze, where dependencies are kept minimal. Then http4s could use that and lose the overhead of Future conversions in its blaze binding. So I'd say http4s would be interested in a production quality implementation here, especially if it could subsume some other extant tasks.

[alexandru]

@djspiewak

I 100% agree this is a concern. This is part of why I was thinking that we bias the reference implementation for composability/formal-coolness. We could make it clear in the documentation that "this works, and it's convenient and cool, but if you're putting code in production you should use Monix's or fs2's Task instead". The problem it solves is really providing an "out of the box" IO type for Cats, which is something that I find myself needing far more often than you would think, as well as a type which is sufficient for http4s to assume.

So lets see such an implementation, might be a good idea, along with these type-classes. Should we starts a cats-effects project and collaborate on it?

[alexandru]

Hi folks, so any further thoughts on this?

Coming up with a solution is a big challenge, but maybe we should solve this piecemeal in order to have progress.

At this point I think the Async and Effect type-classes are very useful at least for enabling seamless conversions between types, without those types knowing about each other.

How can we proceed?

I initiated this a different project for proof of concept, but I can make a PR in Cats if you think a sub-project of cats would be in order. I'd very much like to move this forward.

[alexandru]

Note: I would also like FS2 to depend on it, to have these conversions baked in by default, which is why I think a separate project makes a lot of sense, to avoid the dependency on cats-core.

[edmundnoble]

I'd just like to re-iterate in the strongest possible terms that Future is not a tool for functional programming, and because it is not referentially transparent it already has no business at all even having a Monad instance. Adding an Async instance can only make this worse.

Perhaps I am assuming my point is more obvious than it is: Async.create cannot "sometimes" perform a side-effect and "sometimes" capture that side-effect to be executed later depending on the underlying instance. That is not a useful abstraction, it's an abstraction which will cause a crapload of bugs because Future is not a Task, and you cannot meaningfully abstract over Task and Future. All code that depends on Async will behave entirely differently depending on whether that Async is Future or Task, potentially executing way more effects (or even way less, seeing as Tasks are reusable) than the author intended, using old results silently in some areas and performing extra work in others.

[alexandru]

Updates:

• added Evaluable and Deferrable type-classes
• made that Future instance implement only Effect
• changed license
• changed project name

[larsrh]

I for one would love to see this project in the Typelevel organisation. We have precedent with typeclassic – although that one didn't seem to take of 😦

[alexandru]

Update:

• added effects4s-laws for describing our laws, based on cats-laws / discipline

My plans and hopes:

• to migrate this project to the Typelevel org (will initiate a transfer request)
• to have a stable release, such that @Monix can depend on it
• (optional) to make cats-core depend on effects4s-core, because it is minimal (only describes a couple of simple interfaces) and will stay stable and this would allow Cats to provide instances out of the box for Eval
• (optional) to make FS2 depend on effects4s-core , because these type-classes are just like theirs

Cheers,

[milessabin]

I'd love to see it as part of Typelevel, but I absolutely insist that the project is called "schrodinger"! ;-)

[kailuowang]

@alexandru can we also make a decision on whether to include MonadDefer in this project instead of cats.core?

+1 on "shrodinger" name.

[alexandru]

@milessabin OK, schrodinger it is :-)

I'm renaming it.

[alexandru]

@kailuowang I already included Deferrable in this project. It's no longer a Monad, but that was a concern, due to MTL issues. The laws I described require a Monad instance though.

[mpilquist]

Some miscellaneous thoughts:

• This library is focused on middleware interop, not end user usage. To that end, I think we should have a good interop story for at least 2 libraries. If only Monix supports these type classes, I'm afraid we'll have proliferated another standard with no measurable improvements. https://xkcd.com/927/
• I don't know if FS2 will directly depend on this or not. As of right now, FS2 has no dependencies. We've been debating lifting that restriction to improve the interop story but we haven't reached a conclusion yet. If FS2 doesn't depend directly on this, we could certainly create an interop bridge like we currently have for Scalaz and Cats.
• I don't think cats-core should depend on this library. We need a rock solid, stable cats-core 1.0. I don't think effect capture is that stable yet.

Some implementation comments:

• I don't think UnsafeIO should be in the library if I understand it correctly. It appears to simply add OO-style syntax for an Effect. If that's all it does, I think we should remove it. Alternatively, we could use Simulacrum to add OO syntax to all of the type classes. I think I prefer the former though.
• The Async type class conflicts (name-wise) with fs2.util.Async - the latter is a much more specialized type class though, requiring an implementation of Async#ref.

[alexandru]

@mpilquist

Indeed, my hope is to convince you to depend on it and provide implementations, otherwise it's useless.

I understand the need for no dependencies, but the purpose of this library is to be stable. Once we release 1.0.0, there should be no reason to release another version of the "core", unless we did something stupid that needs to be fixed. Bridges or laws can change, but the core should be set in stone once ready.

This is why I haven't included Simulacrum in the project, because I would prefer for this project to be more stable than Simulacrum's encoding is (as that one can change depending on Scala's evolution).

If you want Async renamed or the removal of UnsafeIO, that's totally fine. I would be thrilled if you accepted to be one of the project's maintainers ❤️

[milessabin]

On the licensing question, I'm be strongly in favour of Apache 2.0 (as I am for all Typelevel projects which aren't GPL).

[djspiewak]

Just wanted to drop a line saying that this hasn't completely fallent off my radar, and I haven't forgotten about my "let's implement a Task!" volunteering. Just been a bit busy this week. Hopefully I'll get back to it this weekend, but generally I think things are moving in a good direction.

Something to spark a little debate so we know where we're going though… This reference Task: how "production ready" do we actually want this to be? Initially I thought "do something formally minimal, composable and cool", but people are going to use it. Like it or not, whatever we do will become the cats Task/IO. The more I think about that, the more I think it's a good thing so long as we are able to get seamless (and by that I don't mean Task { ioa.unsafePerformIO() }) interop with Monix/fs2. fs2 is already parametric in its effect, so basically its own Task is already just a reference implementation. Monix isn't parametric (right?), but I think any Task-like thing is going to be seamlessly convertable. Which is basically a major point of this project. Nobody wants to obsolete anyone's Task, and I think as long as we do our jobs right, we won't.

So basically, are we going to push for production-ish? Or are we going to do something stupid-simple like Cont[IO[Unit], A]? (for reference, I'm almost positive Cont[IO[Unit], A] would be faster than scalaz's Task).

[mpilquist]

@djspiewak You are correct that if we provide a reference implementation, folks will use it and it will become the defacto standard. Hence, I'd shoot for production quality or not provide one at all.

[djspiewak]

@mpilquist A related question… Do we want IO and Task? Or just Task? I tend to think just the latter, but as djspiewak/cont-exp shows, there is still some value (albeit experimental) in having a non-async IO.

[alexandru]

@djspiewak @mpilquist I think at this point a good question would be: what does production ready mean?

From my point of view, Monix's Task is production ready because it gets down and dirty in its implementation, using concurrency primitives defined in the monix-execution sub-project, that would have never happened as part of Cats, because it's basically a complement to scala.concurrent.

For example it provides cache-padded atomic references, which are made to use platform intrinsics, so they work on Java 7, 8 and they'll be optimal on Java 9 as well. These are there to help with shared JVM/JS code, for reusable cancelables, queues and locks, schedulers, etc. I might take this even further for non-blocking queues, as for example monix-reactive already depends on JCTools.org, a bunch of awesome non-blocking queues implementations making use of dark magic basically.

A reference implementation cannot do that, without introducing ugly code in Cats, duplicating the work needlessly, or ending up with a sub-optimal implementation.

And this matters, because performance is a competitive advantage, if you introduce a Task in Cats, it will have to keep up - I was informed for example that the implementation in Scalaz 8 might be inspired by Monix's. And also, based on Task you can move on to other things, other concurrency primitives, like for example MVar, circuit breaker, semaphore and many others are possible.

And in my experience users are glad for these when they have it - not necessarily knowing they are possible. So where do you draw the line if a Task implementation would be in Cats? Because that Task won't be pure and nice at all, or a good community player.

I think we should do this piecemeal though, I really wish for these interfaces to go through.

[djspiewak]

From my point of view, Monix's Task is production ready because it gets down and dirty in its implementation, using concurrency primitives defined in the monix-execution sub-project, that would have never happened as part of Cats, because it's basically a complement to scala.concurrent.

This is essentially where we have a very different perspective on what Task "is". Task, to me, is just an IO that supports asynchronous actions. That's it. It shouldn't have any concurrency stuff, or resource management, or anything really other than that bare minimum. So long as the implementation is clean and transparent, concurrency stuff can be built on top of that.

So in other words, I don't see any need to build something which is a complement to scala.concurrent. fs2 and Monix already do that better than we would want to in this project.

And with respect to performance, it's not that hard to get within rounding error of Monix's Task for most straight-line stuff, especially when concurrency is off the table. Monix's Task is faster than fs2's, but only barely for simple things, and only because of some inlined specialized implementations. I don't have a problem replicating that trick, or not replicating it and just leaving things slightly cleaner. Either way it makes relatively little difference. Monix's real performance wins are elsewhere, and undisputed, and there's no real need for a reference implementation in this project to try to "keep up" with that. Especially if we're not implementing concurrency things.

And also, based on Task you can move on to other things, other concurrency primitives, like for example MVar, circuit breaker, semaphore and many others are possible.

If concurrency stuff (even apply!) is out of scope for cats.effect.Task, then clearly all of this stuff is as well. In my view, if the things you listed (and more) are not possible in a third-party library, built on top of cats.effect.Task, then we did something wrong.

I think we should do this piecemeal though, I really wish for these interfaces to go through.

I agree that we need to nail down the typeclasses. But a reference implementation here is really really important. People care about interop, yes, and pruning down on the constant reinventing of these concepts. But ultimately, I hear a lot more from people about the desperate need for "a cats Task".

[alexandru]

OK then, if you folks are on board with that.

A random thought: Cats already has Eval for IO and it just implemented MonadError as well. And so I would name this reference Task as IO, because on top of Haskell you end up working with IO for async stuff as well, see for example Async#wait. I think Scala's IO should be async, because we don't have the luxury of this wait.

And we don't have to keep familiarity with Scalaz users, the familiarity we should keep is with Haskell and for that purpose an async IO is fine imo.

[alexandru]

And yes, this naming proposal would have the benefit of allowing different implementations trying to innovate on that concept to have a different name 😄

[djspiewak]

@alexandru I'm entirely ok with calling it IO and giving it async structure. I'm a little squeamish about the fact that Eval has a MonadError instance, but that's a freak-out for another day. :-P

Everybody else cool with the "cats Task" being called IO?

[alexandru]

@alexandru I'm entirely ok with calling it IO and giving it async structure. I'm a little squeamish about the fact that Eval has a MonadError instance, but that's a freak-out for another day. :-P

I swear, it wasn't my doing 😝

[djspiewak]

@alexandru THINK OF THE CHILDREN 😇

[fommil]

a common effects library is a good idea. But, perhaps we're taking the naming convention a little too far... I'd be all in favour of pulling a lot of these things back into cats: alleycats, kittens, dogs, schrodinger. It's all fun and all, but maybe it's getting a bit much?