Submitting cats-effect for typelevel incubator membership

[djspiewak]

This is the accompanying typelevel/general issue for typelevel/cats#1617, and a fuller description of the project is there. I'd like to submit cats-effect as a Typelevel project. As this is the accompanying issue, I think it would be fair to treat an ultimate 👎 by the cats contributors as a similar vote on this issue.

1. The only developers are myself, @mpilquist, @rossabaker and @tpolecat, and this was our ultimate goal from the inception of the project
2. There is no mention of the typelevel CoC anywhere on the project at present, but that is easily fixed. Obviously no objections
3. Well, it promotes pure FP with cats in the face of harrowing side-effects, and it does use types. Some of those types are Unit though, so… 2 out of 3 I guess?
4. Readme is present. Scaladoc is very present. Tut docs are not present at all (ironically). Coming soon™

At present, the groupId is set to org.typelevel, simply because I don't think it makes sense to use my personal groupId if the package is cats.effect. I have secretly pushed artifacts to my bintray (mostly so Mike could get a build running), but ultimately I would rather this project sit within the fully-realized Sonatype release process on the main groupId. It is my intent to automate this in Travis, along with the artifact signing, when/if the project is approved.

[larsrh]

Can somebody eli5 to me how this relates to effects4s? Is this orthogonal, redundant or complementary?

[larsrh]

Paging @alexandru

[tpolecat]

My short take is

• cats-effect is an IO data type for Cats, with some effect abstractions defined in terms of the Cats hierarchy. Its goal is to fill a gap in functionality that has been hindering adoption/commitment.
• effects4s is an interop specification for IO-like data types, not in terms of Cats or scalaz or anything else. Its primary use case is to provide generic conversions among these data types.

So to me they're largely orthogonal.

Note that I don't speak for the other cats-effect contributors, who are all asleep. You may get a more nuanced and/or florid response from them tomorrow.

[djspiewak]

@larsrh I'll take a crack at it…

Effects4s is primarily intended to be an agnostic interop specification between multiple "effect-ish" things. Some examples of "effect-ish" things: scalaz.effect.IO, scalaz.concurrent.Task, scala.concurrent.Future, scala.util.Try, monix.Task, fs2.Task, cats.Eval, etc. As the goal of effects4s is to characterize this full space, it does not have a dependency on cats (or anything else) and seeks to be extremely stable and robust.

The primary practical motivation behind effects4s (though certainly not the only one) is the ability to provide dependency-free conversions between effect types. Anything which implements effects4s could define a generic to[F[_]] function which takes another type implementing effects4s, converting to that type. This would generally resolve questions about "can I convert Monix's Task to fs2's?" once and for all, provided that both Monix and fs2 implement effects4s.

cats-effect is a much more narrowly-tailored project. While it does define a hierarchy of typeclasses which is similar in some respects to effects4s, that hierarchy has a cats-core dependency, and we consider that to be a critical characteristic of the project. Additionally, cats-effect defines a concrete IO type which is suitable for production use. A major goal of the project is to be the cats IO: a default for managing side-effects in a pure fashion, but not to the exclusion of other Task implementations which may offer unique features. Monix's Task in particular is quite distinct from IO, and very intentionally so. There is a strong argument for both of them existing, even within the same project. So the concrete IO fills a need rather than crowding out competition.

On a more detailed level, there are some significant differences between the typeclasses defined by effects4s and those defined by cats-effect. Some of this stems from the fact that cats-effect derives from cats-core, so we can use things like MonadError and avoid defining our own Catchable. Some other things though are just design decisions. One example is the fact that none of the cats-effect typeclasses define unsafe functions (e.g. unsafeRunSync() is not in the typeclasses). Instead, if you want to run a generic Effect, you use the runSync function, which returns a concrete IO. In this way, the typeclasses remain lawful and parametric (if we "imbue" IO with magic parametricity w.r.t. side-effects) without losing generic expressivity.

Both cats-effect and effects4s provide a large set of laws which govern the behavior of "effect-ish" types. This is very useful not only in the definition of IO, but also in the definition of other Task-like types which have distinct features (e.g. fs2.Task will use the laws in addition to its own tests). There are significant differences in these laws though, and in fact cats.effect.IO does not obey the effects4s laws, even if it were to define instances of the effects4s typeclasses. Also by cardinality, cats-effect defines more laws, though I'm not sure you can say this is a positive or a negative. It effectively just means that cats-effect is more prescriptive about what sorts of things could be pure effect types (note that Monix's Task and fs2's Task should both satisfy the cats-effect laws, as should Scalaz's Task and its IO, while Scala's Future and Try will not satisfy the laws).

In general, I see the projects as having started from a similar place, but diverging in the design space and now being strongly distinct and even orthogonal. It isn't possible for cats.effect.IO to lawfully implement the effects4s typeclasses, and indeed there is enough conceptual redundancy between the hierarchies that a dependency doesn't make much sense. Conversely, due to its cats-core dependency, cats-effect can never hope to provide a standard abstraction surface which covers players like scalaz or the standard library.

Ultimately, the two projects just address different needs as their primary concerns.

[alexandru]

Hi guys,

The implementation of this project might not necessarily be perfect, but at this point it is a step in the right direction.

From myself this is a 👍 for moving it forward and adding it in Typelevel's Incubator, with these mentions:

• Since this would make sense to be a Typelevel organisation project, it should be moved to the @Typelevel org to facilitate collaboration and preferably to start having Milestone releases (which are going to be published as org.typelevel)
• Please consider integrating Effects4s in it as well ... we can negotiate the details, maybe make Effects4s lighter and more compatible with what you want in cats-effect, maybe renaming it - but as another anecdote, just the other day I got a question at Scala Days from somebody asking me to add Twitter Future interoperability in Monix, which basically means conversions ... users really want easy conversions without pain and for example I'm sure we can convince the Twitter engineers to depend on Effects4s

Also, I would like to be one of the maintainers of cats-effect, if you don't mind. Totally optional, if you think our opinions are so diverging 😇

Cheers,

[djspiewak]

Since this would make sense to be a Typelevel organisation project, it should be moved to the @Typelevel org to facilitate collaboration and preferably to start having Milestone releases (which are going to be published as org.typelevel)

Absolutely 100% yes. When/if this project is accepted, the main repo moves under the typelevel org. Though, regarding milestones, I'm rabidly opposed to that particular release process. We can have a meta-discussion on that later though. :-)

Please consider integrating Effects4s in it as well

I think that's a longer conversation. It was definitely something that I looked very seriously into. The problems at present are multi-faceted.

• The effects4s laws are incompatible with IO. In other words, IO cannot lawfully implement the effects4s typeclasses. Neither can fs2.Task or scalaz.concurrent.Task. So that's a bit of a problem, though not (I think) one that should be hard to resolve. scala.concurrent.Future also cannot lawfully implement the effects4s laws, so basically right now, the only Task-like thing I can find which is lawful is Monix's.
• At present, effects4s provides a lot of stuff; its typeclass hierarchy is actually more elaborate than that of cats-effect! Since we're proposing this would be another upstream dependency, it would really need to be very very lean to justify that kind of positioning. I think we would need to be crystal clear about what problems it is solving, and include exactly and only what is necessary to solve those problems.
• In the event that effects4s provides a larger set of typeclasses, integration starts to become a problem, since you don't want effects4s to depend on cats-core, but you absolutely need things like Monad to even define laws, so you end up in a bit of a bind w.r.t. how the hierarchy is laid out. This problem is compounded considerably in downstream projects which do depend on cats-core, such as cats-effect.
  • Related to this is the fact that the cats-effect typeclasses do not abstract over any unsafe functions, and I believe very strongly that this is the right approach from a parametricity standpoint. Unfortunately, the only way we're able to do this is to have the typeclasses depend directly on IO. This isn't a problem for cats-effect, but clearly it's not a trick that can be used in effects4s.

Anyway, I don't think these are insurmountable problems, but as I said in my earlier comment: the design goals of effects4s and cats-effect have diverged quite a bit, and I'm not sure how compatible they are at this point in time. Definitely worth having a long discussion and doing the design work to find out though.

Also, I would like to be one of the maintainers of cats-effect, if you don't mind.

I don't mind at all. When code is exclusively getting in via pull requests (rather than committers pushing master), then diverging opinions become a project strength rather than a project weakness. I hold myself to that as well. Now that the project is open, I intend to pull request anything but the most trivial of things so other people can weigh in and refute. It's not my project anymore, IMO, and hasn't been for a while.

[tpolecat]

It seems to me that there is sufficient support to move forward with this. @alexandru has stated that his concrete concerns are not blockers, and he is now a project contributor so I think we're in great shape. cc: https://github.com/typelevel/cats/issues/1617

[larsrh]

Welcome, @djspiewak! Next steps:

1. Please post on this issue, similarly to the previous requesters there.
2. I've invited you as a temp admin on the organisation so that you can transfer the repository here.

[djspiewak]

1. Done
2. Will do once back at laptop Done

[larsrh]

Please send a PR to the website adding the project.