kitlangton / parallel-for

Automatically parallelize your for-comprehensions at compile time.
Apache License 2.0
131 stars 10 forks source link

Support abstract effect type (with `cats.Monad` & `cats.Parallel` instances) #10

Open jilen opened 2 years ago

RobertoUa commented 2 years ago

is this coming? current cats-effect support is not really useful for me because it works only with IO

masonedmison commented 2 years ago

I started poking around on this, and as others have alluded to (#4), this doesn't seem to be a trivial thing to add support for. Merely making Parallelizable1 parametric on the effect type with a Parallel capability is trivial, but when swapping this in with the Parallelizable1 with concrete IO

  implicit def forParallel[F[+_]: Parallel]: Parallelizable1[F] = new Parallelizable1[F] {
    override def zipPar[A, B](left: F[A], right: F[B]): F[(A, B)] =
      (left, right).parTupled
    override def flatMap[A, B](fa: F[A], f: A => F[B]): F[B] =
      Parallel[F]
        .flatMap
        .flatMap(fa)(f)
    override def map[A, B](fa: F[A], f: A => B): F[B] =
      Parallel[F]
        .flatMap
        .map(fa)(f)
  }

  // implicit val ioPar1: Parallelizable1[IO] = new Parallelizable1[IO] {
  //   override def zipPar[A, B](left: IO[A], right: IO[B]): IO[(A, B)] = left.both(right)
  //   override def flatMap[A, B](fa: IO[A], f: A => IO[B]): IO[B]      = fa.flatMap(f)
  //   override def map[A, B](fa: IO[A], f: A => B): IO[B]              = fa.map(f)
  // }

  type IOP[-_, +_, +A] = IO[A]

  implicit def convertIO(implicit p1: Parallelizable1[IO]): Parallelizable[IOP] = new Parallelizable[IOP] {
    override def zipPar[R, E, A, B](left: IOP[R, E, A], right: IOP[R, E, B]): IOP[R, E, (A, B)] = p1.zipPar(left, right)
    override def flatMap[R, E, A, B](fa: IOP[R, E, A], f: A => IOP[R, E, B]): IOP[R, E, B]      = p1.flatMap(fa, f)
    override def map[R, E, A, B](fa: IOP[R, E, A], f: A => B): IOP[R, E, B]                     = p1.map(fa, f)
  }

yields a lovely exception during macro exception.

exception during macro expansion: 
scala.reflect.macros.TypecheckException: effect is not an enclosing class
    at scala.reflect.macros.contexts.Typers.$anonfun$typecheck$3(Typers.scala:44)
    at scala.reflect.macros.contexts.Typers.$anonfun$typecheck$2(Typers.scala:38)
    at scala.reflect.macros.contexts.Typers.doTypecheck$1(Typers.scala:37)
    at scala.reflect.macros.contexts.Typers.$anonfun$typecheck$7(Typers.scala:50)
    at scala.reflect.internal.Trees.wrappingIntoTerm(Trees.scala:1891)
    at scala.reflect.internal.Trees.wrappingIntoTerm$(Trees.scala:1888)
    at scala.reflect.internal.SymbolTable.wrappingIntoTerm(SymbolTable.scala:28)
    at scala.reflect.macros.contexts.Typers.typecheck(Typers.scala:50)
    at scala.reflect.macros.contexts.Typers.typecheck$(Typers.scala:32)
    at scala.reflect.macros.contexts.Context.typecheck(Context.scala:18)
    at scala.reflect.macros.contexts.Context.typecheck(Context.scala:18)
    at parallelfor.internal.Macro.parallelizeImpl(Macro.scala:106)

^^ to be exact. I am more than happy to dig deeper in this but would appreciate any advice in how to attack this (I assume there will need to be modifications to the macro itself) since macros in scala 2 are a bit new to me.