portals-project / portals

Portals is a framework for stateful serverless apps, unifying dataflow streaming with actors
https://www.portals-project.org/
Apache License 2.0
19 stars 2 forks source link

Revisit contextual function type ordering for UDFs #153

Open jspenger opened 1 year ago

jspenger commented 1 year ago

One of the pervasive issues in Portals has to do with context functions. For example, this is the current implementation of the MapTask in Portals:

private[portals] case class MapTask[T, U](f: TaskContextImpl[T, U, _, _] => T => U)
    extends BaseTask[T, U, Nothing, Nothing]:
  override def onNext(using ctx: TaskContextImpl[T, U, Nothing, Nothing])(t: T): Unit = ctx.emit(f(ctx)(t))
end MapTask // trait

However, this is the corresponding method from the factory/builder:

def map[T, U](f: MapTaskContext[T, U] ?=> T => U): GenericTask[T, U, Nothing, Nothing] =
    MapTask[T, U](ctx => f(using ctx))

Another thing is awkward is passing a contextual function f as an argument to anything: it will always cause an error that it cannot find a contextual value, and instead one has to write something like c ?=> f(using c) regardless.

The awkwardness stems from that a case class cannot have as a parameter a contextual function type. So, at an earlier point in the implementation, the choice was made to do it as above. Now, it is perhaps a good time to revisit this principle, as after some more recent trials other potential work-arounds have surfaced.

One option could be to have the contextual parameter not as the first parameter, like the following:

private[portals] case class MapTask[T, U](f: T => TaskContextImpl[T, U, _, _] ?=>  U) ...

The second option could be to have it as a second parameter list, like the following:

private[portals] case class MapTask[T, U]()(f: TaskContextImpl[T, U, _, _] ?=> T => U) ...

// Rant over

We should revisit this at some point.