Analogue to await, but for Resource (instead of IO)

[sideeffffect]

It would be great, if there were support for using Resource with cats-effect-cps. Could look something like this

import cats.effect.cps._

async[IO] {
  val results1 = talkToServer("request1", None).await
  IO.sleep(100.millis).await

  // openResource: (String, Option[Data]) => Resource[IO, Data]
  val results2 = openResource("request2", Some(results1.data)).awaitResource

  if (results2.isOK) {
    writeToFile(results2.data).await
    IO.println("done!").await
    true
  } else {
    IO.println("abort abort abort").await
    false
  }
}

awaitResource is pretty bad name for this, but can't think of anything better at the moment. Over at dotty-cps-async, I recommended ! for IO and ~ for Resource: https://github.com/rssh/dotty-cps-async/discussions/43#discussioncomment-1010773

The motivation is that using Resource is a bit unergonomic, especially if there are many of them, because it requires the programmer to nest the blocks in use.

Related discussion about Resource at https://github.com/rssh/cps-async-connect/issues/2#issuecomment-881889247

/cc @rssh

[rssh]

Btw, currently, in cps-async-connect resources is supported in two ways:

1. using an extension to Resource.type, such as Resource mimic scala-core 'Using'

   Resouce.using(r1,r2,r3) {
   ....
   }

   This can be easily ported to cats-effect-cps, I can submit a pull request if maintainers will agree.

2. using asyncScope. which holds monad [X]=>>Resource[IO,X]. internally, but exposed as IO to the external world.

   asyncScope[IO] {
   r1 = await( open(...) )
   r2 = await( open(...) )
   ...
   }

   It can be nicely generalized if we assume, that async[IO]. holds not IO, but some Control[IO]. and define the syntax for embedding Resource (and maybe some other constructions) into Control[IO]. But I'm not sure, that the representing async[Control[IO]]{ ... }.effect as async[IO] is the correct decision, may-be better have yet-one syntax.

[Baccata]

Hello !

@djspiewak will be able to put it more eloquently than I can, but having some magic awaitResource syntax that unlifts Resource into IO is probably not a great idea, because it downplays the importance of managing lifecycles properly. It also hides the compositional nature of Resources (which is something we really want to promote).

The motivation is that using Resource is a bit unergonomic, especially if there are many of them, because it requires the programmer to nest the blocks in use.

That's actually an anti-pattern (which is why it probably feels awkward to you 😉) : best practice would have Resources be composed together using monadic combinators (or for-comprehensions), and calls to use be minimised to a single one.

If you really want to mix IO and Resource into the same control-flow construct, the better to go at it is rather to lift IO into Resource, and have the control-flow work on Resource (which is kinda similar, to an extent, to what @rssh suggests)

type R[A] = Resource[IO, A] 
val computeR : R[Boolean] = async[R]{
  val results1 = talkToServer("request1", None).to[R].await
  IO.sleep(100.millis).to[R].await

  // openResource: (String, Option[Data]) => Resource[IO, Data]
  val results2 = openResource("request2", Some(results1.data)).await

  if (results2.isOK) {
    writeToFile(results2.data).to[R].await
    IO.println("done!").to[R].await
    true
  } else {
    IO.println("abort abort abort").to[R].await
    false
  }
} 

val compute : IO[Boolean] = computeR.use(result => ???) 

For the record, I personally see cats-effect-cps as a gateway for people who come from languages that support the async/await style than a mean to completely hide the paradigm under the carpet via syntactic sugar. We want you to build some intuition for how IO and Resource relate to each other 😺

[sideeffffect]

Hello @Baccata !

best practice would have Resources be composed together using monadic combinators (or for-comprehensions), and calls to use be minimised to a single one

That's also not great from usability point of view. You then have to do a match on possibly many things in big tuple and then have a nested use block (even though it's just one level).

val res = for {
  a <- openA
  b <- openB
  c <- openC
  d <- openD
  e <- openE
  f <- openF
  ...
} yield (
  a,
  b,
  c,
  d,
  e,
  f,
  ...
)

val io = for {
  z <- zzz
  x <- res.use {
    case (
      a,
      b,
      c,
      d,
      e,
      f,
      ...
    ) =>
      ...
  }
  w = www
  y <- yyy
} yield y

If you really want to mix IO and Resource into the same control-flow construct, the better to go at it is rather to lift IO into Resource

Yes, that's another option. But it's still not great with regards to programmer friendliness. Those lifts/.to[Resource[IO, *]]s feel like warts. And you're forced to work at the level of Resource instead of IO -- simpler and easier to teach is to work at the level of IO.

Ideally,it could work like something like this (see my proposal at dotty-cps-async):

val io = lift[IO] {
  val z = ! zzz
  val a = ~ openA
  val b = ~ openB
  val c = ~ openC
  val d = ~ openD
  val e = ~ openE
  val f = ~ openF
  val x = ! ...
  val w = www
  val y = ! yyy
  y
}

Have you ever worked with F#? It has a language feature called Computation expressions and it has the keyword use! which to be used for binding resources (whereas let! is for regular "monad" values, like IO in our case and let is for plain binding of variables). And it works really great there and I don't see any reason why we couldn't have something analogous in Scala too. The example from above would look in F# like this

let io = async {
  let! z = zzz
  use! a = openA
  use! b = openB
  use! c = openC
  use! d = openD
  use! e = openE
  use! f = openF
  let! x = . . .
  let w = www
  let! y = yyy
  return y
}

We want you to build some intuition for how IO and Resource relate to each other

Definitely! I'm not suggesting to hide the difference between Resource and IO or how they relate to each other. I would just like if working with Resource were simpler :wink:

[armanbilge]

Just want to briefly chime in with the following suggestion:

Applicative[Resource[IO, *]].tupled3(r1, r2, r3).use { case (r1, r2, r3) =>
  // await and prosper!
}

@Baccata I assume you have no objections to this, and @sideeffffect is this what you are looking for, just with maybe some shortcut/nicer syntax for the Applicative[Resource[IO, *]].tupled part?

[sideeffffect]

Hi @armanbilge ! Actually no :smile_cat: In my eyes, using use explicitly and then having to do a pattern match on a (potentially) big tuple and then have an indented block is user unfriendly -- it feels like using explicit flatMaps.

What I'm looking for is something like

val io = lift[IO] {
  val z = ! zzz
  val a = ~ openA
  val b = ~ openB
  val c = ~ openC
  val d = ~ openD
  val e = ~ openE
  val f = ~ openF
  val x = ! ...
  val w = www
  val y = ! yyy
  y
}

[armanbilge]

@sideeffffect ok, what if we magicked that syntax down to something like:

Using(resourceA, resourceB, resourceC) { (a, b, c) => 
  // await and prosper
}

The repetition of (a, b, c) still is the problem I suppose?

[sideeffffect]

:slightly_smiling_face: yes, it's a bit better. But not substantially. One still has to have a one more indented block. I would really like us to get to something similar to what F# programmers enjoy (I encourage you to have a look at the Computation expressions tutorial I mentioned earlier).

Just as we don't want to write

aio.flatMap { a =>
  bio(a).flatMap { b =>
    cio(b).flatMap { c =>
      ...
    }
  }
}

we shouldn't want to write

ares.use { a =>
  bio(a).flatMap { b =>
    cres(b).use { c =>
      ...
    }
  }
}

[Baccata]

And you're forced to work at the level of Resource instead of IO

Working (or being forced to work) at the level of resource whenever any kind of lifecycle is involved is desirable (imho), and the fact that one might not like the aesthetics of it, or that it "feels like a wart" to some, is a personal preference.

I'm not suggesting to hide the difference between Resource and IO or how they relate to each other

Unfortunately, the proposal does that, even if it's not intended 😅. You're also saying "simpler and easier to teach is to work at the level of IO", which kinda implies that the complexity of lifecycle management ought to be hidden under the carpet.

That being said, I'd be more inclined to see the proposal in a positive light if it was accompanied by a pull-request (even a partially-working one).

[djspiewak]

So as a quick prelude to all of this… It's already possible to get relatively close to what you want:

import cats.effect.cps._

async[Resource[IO, *]] {
  val results1 = talkToServer("request1", None).await
  IO.sleep(100.millis).toResource.await

  // openResource: (String, Option[Data]) => Resource[IO, Data]
  val results2 = openResource("request2", Some(results1.data)).await

  if (results2.isOK) {
    writeToFile(results2.data).toResource.await
    IO.println("done!").toResource.await
    true
  } else {
    IO.println("abort abort abort").toResource.await
    false
  }
}

In general, I really agree with @Baccata here: the distinction between Resource and IO isn't something we want to hide. The whole point of Resource is to explicitly declare your lifecycles and cause them to behave in a fashion which is very different from how IO behaves under normal circumstances. Blurring the lines between the two, even syntactically, will carrot people into very sub-optimal constructions that hold onto resources far longer than they probably intended. It also reduces the incentive to think carefully about where your resources are scoped and why.

There's obviously a difficult line to walk here. After all, we are trying to make things easier for our users, not harder. However, I believe very firmly that we should always try to make the right path easy and the wrong path difficult, even when we have the ability to make both of them easy. This comes very close to falling into that kind of category, so I would generally vote :-1:.

[sideeffffect]

Unfortunately, the proposal does that, even if it's not intended :sweat_smile:. You're also saying "simpler and easier to teach is to work at the level of IO", which kinda implies that the complexity of lifecycle management ought to be hidden under the carpet.

Blurring the lines between the two, even syntactically

val io = lift[IO] {
  val z = ! zzz
  val a = ~ openA
  val w = www
  val y = ! yyy
  y
}

For the record, in my eyes, this doesn't blur the lines nor hide things under the carpet (! vs ~ vs nothing) and that's probably why I think this proposal is a good idea. (I also haven't seen any complaints that F# with its computation expressions makes people confused about resource management.)

carrot people into very sub-optimal constructions that hold onto resources far longer than they probably intended

I think I see what you mean here. From my experience with Resource, this is actually very often OK -- most uses of Resources don't need to be super micro-optimized. It's good that it's possible when it's actually necessary, but I don't think that happens very often. So I think that this proposal honors the principle making the common usecase easy while making the complicated usecase possible.

But maybe the premises that I'm working with (the distinction still being obvious, not muddying the workings of Resource, not needing microoptimized Resource often) are wrong.

[djx314]

I think it's a hard problem. Just show a sample

// trait Useable[T]

// val r: Resource[IO, Connection] = ???
// val data1: IO[Int] = ???
// val insertAction: IO[Json] = async2[Resource[IO, *], IO].to[Useable, Id] { // only the last async type parameter can effect the result type
//  val resource1: Useable[Connection] = r.await1
//  val result1: Int = data1.await2
//  val result2: List[Int] = resource1.use { r =>
//    r.dealWith(result1).await
//  }
//  result2.asJson
//}

I can't handle the relationship between these types anyway.

[djx314]

scalax/ce-nat-sample

@sideeffffect @djspiewak It's a sample part of ghdmzsk/ghdmzsk

It use nat to abstract cats-effect. And I think it can solve this question. But it's a hard change.

This code expresses that, not point to Monad[F].

This issue is because cats-effect-cps is point to Context[S].anyMethod(S => Context[T]): Context[T]

Change the point to AnyObject[S].anyMethod(S => Context[T]): Context[T]

Then set the type Context[T] = cats.effect.IO[T]

Now IO[S].flatMap(S => IO[T]): IO[T] and Resource[IO, S].use(S => IO[T]): IO[T] is the same abstraction by different lifting.

And AnyObject[S].use(S => IO[T], moreParameter), AnyObject[S].then(IO[T]) is easy change to the abstraction by a lifting like resource_use(resourceFuncTag, moreParameter) and so on.

[djx314]

It means that change F[T].flatMap(T => F[S]): F[S] to G[T].flatMap(T => F[S]): F[S] Because cats.effect.IO is the center of cats-effect

[djx314]

@djspiewak Sorry for the error point.

I voted not to implement this function.

https://github.com/scalax/simple/blob/58140ab6332f83a50638cb6e77a17a04e3197a46/modules/main/simple-core/shared/src/main/scala/net/scalax/simple/core/Core3.scala#L3-L9

trait Core3_1[M[_], G[_]] {
  def apply[T]: M[T] => G[T]
}

trait Core3_2[F[_]] {
  def apply[A, B]: F[A] => F[B]
}

cats is just Core3_1 and Core3_2 and add some dlc(steam's dlc), like:

cats.~> is Core3_1 itself.
unsafeRun is Core3_1[IO, cats.Id]
pure is Core3_1[cats.Id, IO]
flatMap is (A => F[B]) => Core3_2[F].apply[A, B]
map is (A => B) => Core3_2[F].apply[A, B]
Application.<*> is F[A => B] => Core3_2[F].apply[A, B]
Resource.eval is Core3_1[IO, Resource[IO, *]]

So cats-effect-cps is a lib that just point Core3_2, if nobody can put Core3_1 and Core3_2 as The One. That use two F[_] Context to implement cps can lead to exponential explosion liked codegen.