Monocle's polymorphic optics don't work properly with case classes

kenbot commented 3 years ago

Case classes in Scala

The copy methods that Scala generates for polymorphic case classes are quite sophisticated.

For instance, for case class Box[A](a: A), the copy method will look something like: def copy[B](a: B): Box[B]. Notice that it might return a different type depending on the input.

It is very flexible; consider case class FBox[F[_], A](fa: F[A]); the copy method will look like def copy[G[_], B](fa: G[B]): Box[G,B]. The user never needs to specifiy the type parameters, the typechecker will just figure it out from the given argument.

The trouble with Monocle's POptics

Monocle's polymorphic optics such as PLens[S,T,A,B] cannot reproduce this behaviour, because the T and B arguments are locked in at the time the optic is constructed , but to access the case class copy behaviour (ie replace, modify, etc) they need to be chosen at that point of use.

Taking the Box[A]/def copy[B] example, it's inconceivable that a user would construct a separate optic instance for every concrete combination of A and B. Given how important case classes are in Monocle's user story, honestly I would be surprised if anyone in userland has used Monocle's polymorphic optics in anger at all. This is very embarrassing!

Why us? What about Haskell?

Polymorphic updating "just works" in Haskell's Control.Lens, because:

It is a lazy language;
There is no difference between a value and a function;
Control.Lens has no "Lens" type as such; everything is just built out an elaborate series of interlocking functions;
Actions like set exist at the top level and are passed the unresolved optic functions; ie not locked inside the lens object that needs to be concretely constructed before the action can be executed.

The t and b type arguments are not "frozen" at construction time; the optics are functions that retain their uncommitted type variables. Only when set (for example) is finally called does it all collapse into known concrete types.

set _2 42 ("Hello", "world")
-- ==> ("Hello", 42)

What about Kotlin?

Kotlin's data classes have a generated copy method, but it does not support polymorphic behaviour. I haven't checked arrow-kt's behaviour, but failing to accommodate polymorphic behaviour would still leave it capable of handling the entirety of Kotlin's expected data class behaviour. In Scala though, we have a higher standard to uphold!

What about other Scala optics solutions?

Last time I checked (albeit four years ago), none of the other lens libraries (eg the Shapeless one, Goggles, the fluent DSL one, etc) supported proper polymorphic behaviour either. Probably because it is hard!

What can we do?

At a rough sketch, to support case classes, Monocle might need some concept of an optic looking something like this:

trait Lens[S,A] {
  type T[_] // Type function to get the big answer from the little answer
  def get(s: S): A
  def replace[B](b: B): S => T[B]
}

def boxLens[A]: Lens[Box[A], A] = new Lens {
  type T[X] = Box[X]
  def get(s: Box[A]): A = s.a
  def replace[B](b: B): Box[A] =>Box[B] = _.copy(a = b)
}

boxLens.replace(42)(Box("hello"))
// ==> Box(42)

Something like that would work just fine for a polymorphic case class; but not every case is a polymorphic case class. I suspect that there are many valid lenses where T cannot be expressed as a simple type function from B. So then we have a thicket of new questions - would this style (or similar) replace the usual optics, or live beside them? How does it compose? Do the laws work the same? etc

Anyway. We need to do a bit of thinking here @julien-truffaut @yilinwei ...

julien-truffaut commented 3 years ago

Pinging @Odomontois (author of tofu-optics) which supports polymorphic optics.

I hope you don't mind. I thought you might find this topic interesting and I remember you mentioned you use polymorphic optics in Scala 2.

kenbot commented 3 years ago

Oh cool, I hadn't heard of tofu optics, nice work @Odomontois!

If I'm not mistaken, this library will have the same problem, as the optics classes such as PContains[-S, +T, +A, -B] have all the types baked into the constructor, and modifier methods like set and update are instance methods attached to the pre-constructed optic.

kenbot commented 3 years ago

Very happy to report that I have been a touch melodramatic. No need to rewrite anything. Here's what we can do:

Polymorphic optics need to be defined as polymorphic functions that return optics, rather than the direct optics.
The Scala compiler does it all just right; it will defer committing to choosing the types until the very last second, so that it all just clicks into place when we finally call replace, etc.
This works with composition expressions too, it will happily defer choosing the types for the whole thing.
In our generated stuff, like Focus and whatever we end up doing for #7, we need to be generating methods like the Pair.a and Pair.b methods in the example below. They should plug in fine with the other optics.

You can paste this code straight in Scastie:

trait Lens[S,T,A,B] { self => 
  def get(s: S): A
  def replace(b: B): S => T

  final def >>>[C,D](lens: Lens[A, B, C, D]): Lens[S,T,C,D] = new Lens {
    override def get(s: S): C = lens.get(self.get(s))
    override def replace(d: D): S => T = 
      s => self.replace(lens.replace(d)(self.get(s)))(s)
  }
}

case class Pair[A,B](a: A, b: B)

object Pair {
  def a[A,B,A1]: Lens[Pair[A,B],Pair[A1,B],A,A1] = new Lens {
    def get(pair: Pair[A,B]): A = pair.a
    def replace(x: A1): Pair[A,B] => Pair[A1,B] = _.copy(a = x)
  }

  def b[A,B,B1]: Lens[Pair[A,B],Pair[A,B1],B,B1] = new Lens {
    def get(pair: Pair[A,B]): B = pair.b
    def replace(x: B1): Pair[A,B] => Pair[A,B1] = _.copy(b = x)
  }
}

@main
def main(): Unit = {
  val pairs = Pair(Pair("hi", Pair(true, false)), 4)

  val result = (Pair.a >>> Pair.b >>> Pair.b).replace(42)(pairs)
  println(result)
}
// ==> Pair(Pair(hi,Pair(true,42)),4)

optics-dev / Monocly