Document how to do Semi-Auto Derivation with Typeclasses

deusaquilus commented 4 years ago

NOTE This issue has been modified several times as I have learned more about Typeclass derivation. Please see the Edit sections below for latest info.

In Quill, we frequently use the following pattern:

For some given type T (note, we do not have an actual instance of T yet!)

If T is not a product, check if an Encoder[T] can be summoned. If it can, summon it and write it out to a data-structure.
If T is a product, get the child elements of T (typically from the constructor params), find their types and recurse.

I am trying to reproduce this kind of behavior with mirrors as I understand it is no longer possible to do things like tpe.members.collect. The trouble is, it isn't working.

minimized code

  // Let's say I have some Fooify instances for my leaf-node types
  given Fooify[Int] = new Fooify[Int] with
    def fooify: String = "IntFoo"

  given Fooify[String] = new Fooify[String] with
    def fooify: String = "StringFoo"

  // My main recursion loop is here:
  inline def processType[Elems <: Tuple]: List[String] =
    inline erasedValue[Elems] match {
      case _: (elem *: rest) =>
        val innerMirror =
          summonFrom {
            case m: Mirror.Of[elem] =>
              m match {
                case mp: Mirror.ProductOf[elem] => Some(mp)
                case _ => None
              }
            case _ => None
          }
        innerMirror match {
          case Some(m) => processType[m.MirroredElemTypes] ++ processType[rest]
            summonFrom {
              case fe: Fooify[elem] => List(fe.fooify) ++ processType[rest]
            }
        }

      case _ => Nil
    }

  // Then I kick the whole process off like this:
  inline def fooifyType[T]: List[String] = {
    summonFrom {
      // it's a container type
      case m: Mirror.Of[T] =>
        println("Product mirror found: " + m)
        m match {
          case mp: Mirror.ProductOf[T] => processType[m.MirroredElemTypes]
        }
      // it's a single-value type
      case _ =>
        summonFrom {
          case fe: Fooify[elem] => List(fe.fooify)
        }
    }
  }

  // Then I execute this function like so:
  @main def runFooify() =
    println(fooifyType[SuperPerson])

expectation

My expectation is for this code to work properly and go through the children of some type T. Instead, there are several very odd behaviors.

Firstly, I get an implicit resolution error because for some reason, it will match Fooify[T] for any T.

[error] 10 |  println(fooifyType[SuperPerson])
[error]    |          ^^^^^^^^^^^^^^^^^^^^^^^
[error]    |ambiguous implicit arguments: both method given_Fooify_Int in object SimpleMacro and method given_Fooify_String in object SimpleMacro match type simple.SimpleMacro.Fooify[elem]
[error]    | This location contains code that was inlined from SimpleMacro.scala:62
[error]    | This location contains code that was inlined from SimpleMacro.scala:60
[error]    | This location contains code that was inlined from SimpleMacro.scala:60
[error]    | This location contains code that was inlined from SimpleMacro.scala:75
[error] one error found

Say I only keep Fooify[String] around and comment out the other Fooify, then, the following error happens:

[error] assertion failed: unresolved symbols: type elem(line 56), type elem(line 56), type elem(line 56), type elem(line 56), type elem(line 56), type elem(line 56), type elem(line 56) when pickling /home/alexander/git/dotty/dotty_test/src/main/scala/simple/SimpleMacroMain.scala
[error] dotty.DottyPredef$.assertFail(DottyPredef.scala:17)
[error] dotty.tools.dotc.core.tasty.TreePickler.pickle(TreePickler.scala:704)
[error] dotty.tools.dotc.transform.Pickler.run$$anonfun$10$$anonfun$8(Pickler.scala:63)
[error] dotty.runtime.function.JProcedure1.apply(JProcedure1.java:15)
[error] dotty.runtime.function.JProcedure1.apply(JProcedure1.java:10)
[error] scala.collection.immutable.List.foreach(List.scala:305)
[error] dotty.tools.dotc.transform.Pickler.run$$anonfun$2(Pickler.scala:87)
[error] dotty.runtime.function.JProcedure1.apply(JProcedure1.java:15)
[error] dotty.runtime.function.JProcedure1.apply(JProcedure1.java:10)
[error] scala.collection.immutable.List.foreach(List.scala:305)
[error] dotty.tools.dotc.transform.Pickler.run(Pickler.scala:87)

Edit:

As it turns out, you can implement this pattern using generic derivation so this is just a documentation issue. See further below for a better example:

(This is how to do the same thing with typeclass derivation)

object Fooify {
  // Let's say I have some Fooify instances for my leaf-node types
  given Fooify[Int] = new Fooify[Int] with
    def fooify: String = "IntFoo"

  given Fooify[String] = new Fooify[String] with
    def fooify: String = "StringFoo"

  // My main recursion loop is here:
  inline def processType[Elems <: Tuple]: List[String] =
    inline erasedValue[Elems] match {
      case _: (elem *: rest) =>
        val innerMirror =
          summonFrom {
            case m: Mirror.Of[elem] =>
              m match {
                case mp: Mirror.ProductOf[elem] => Some(mp)
                case _ => None
              }
            case _ => None
          }
        innerMirror match {
          case Some(m) => processType[m.MirroredElemTypes] ++ processType[rest]
            summonFrom {
              case fe: Fooify[elem] => List(fe.fooify) ++ processType[rest]
            }
        }

      case _ => Nil
    }

  inline def derived: Fooify[T]: List[String] =
    summonFrom {
      case ev: Mirror.Of[T] =>
        inline ev match {
          case mp: Mirror.ProductOf[T] => processType[m.MirroredElemTypes]
        }
    }

  object Implicits {
    implicit inline def fooify[T]: Fooify[T] = Fooify.derived
  }
}

Then all you need to do is this:

@main def myTest() = {
  case class Person(name: String, age: Int)
  import Fooify.Implicits._
  summon[Fooify[Person]].fooify
}

More Edit:

As it turns out this pattern is well known as auto-derivation. I think it should definitely be documented on the typeclass-derivation page.

odersky commented 4 years ago

It would be good to minimize this further. Right now it's hard to tell whether this is a bug or not.

nicolasstucki commented 4 years ago

The example is incomplete and cannot be compiled. @deusaquilus could you create a version that contains all the code needed to compile it and try to minimize it a bit more.

deusaquilus commented 4 years ago

Please give me a couple days and I will reproduce the issue in Scastie

deusaquilus commented 4 years ago

As it turns out, what I tried doing here is possible by using derived instead. The following example of Eq[T] has been modified to show this.

(A bit more detail: Basically, the whole pattern of "summon a Mirror for T, otherwise T is a leaf node" is completely superseded by Dotty's generic-derivation mechanism. The example I posted at the top can be fulfilled by generic derivation. The only problem was, from the Documentation I thought it was necessary to provided typeclass instances via class MyClass derives TC or given TC[MyClass]. As it turns out, if you write your derived method in a particular way, this is not needed.)

package example.eqalternate

import scala.deriving._
import scala.quoted._
import scala.quoted.matching._
import scala.compiletime.{summonFrom, erasedValue}

trait Eq[T] {
  def eqv(x: T, y: T): Boolean
}

object Eq {
  given Eq[String] {
    def eqv(x: String, y: String) = x == y
  }
  given Eq[Int] {
    def eqv(x: Int, y: Int) = x == y
  }

  def check(elem: Eq[_])(x: Any, y: Any): Boolean =
    elem.asInstanceOf[Eq[Any]].eqv(x, y)

  def iterator[T](p: T) = p.asInstanceOf[Product].productIterator

  def eqSum[T](s: Mirror.SumOf[T], elems: List[Eq[_]]): Eq[T] =
    new Eq[T] {
      def eqv(x: T, y: T): Boolean = {
        val ordx = s.ordinal(x)
        (s.ordinal(y) == ordx) && check(elems(ordx))(x, y)
      }
    }

  def eqProduct[T](p: Mirror.ProductOf[T], elems: List[Eq[_]]): Eq[T] =
    new Eq[T] {
      def eqv(x: T, y: T): Boolean =
        iterator(x).zip(iterator(y)).zip(elems.iterator).forall {
          case ((x, y), elem) => check(elem)(x, y)
        }
    }

  inline def summonInstance[T]: Eq[T] = summonFrom {
    case t: Eq[T] => t
  }

  inline def summonAll[T <: Tuple]: List[Eq[_]] = inline erasedValue[T] match {
    case _: Unit => Nil
    case _: (t *: ts) => summonInstance[t] :: summonAll[ts]
  }

  inline def derived[T]: Eq[T] =
    summonFrom {
      case ev: Mirror.Of[T] =>
        inline ev match {
          case s: Mirror.SumOf[T]     => eqSum(s, summonAll[s.MirroredElemTypes])
          case p: Mirror.ProductOf[T] => eqProduct(p, summonAll[p.MirroredElemTypes])
        }
    }

}

object Macro_4 {
  implicit inline def eqGen[T]: Eq[T] = Eq.derived

  inline def [T](x: =>T) === (y: =>T): Boolean = {
    val eq = summon[Eq[T]]
    eq.eqv(x, y)
  }
}

Then you can just do:

package example.eqalternate

@main def eqTest() = {
  case class Address(street: String) // Hurrah! Don't need to do 'derived'
  case class Person(name: String, address: Address) // Hurrah! Don't need to do 'derived'

  import Macro_4._

  println( Person("Joe", Address("123")) === Person("Joe", Address("123")) )
}

I did not know that the above pattern was possible. Please add it to the documentation page and/or make a regression test for it.

biboudis commented 4 years ago

Now I see! Where do you think its best to mention this in the documentation so that it is clear? Can you leave a suggestion on #8011 as a comment? And I will incorporate it in that PR! Thx! @deusaquilus

deusaquilus commented 4 years ago

So a couple of notes about derivation.md.

The given match construct doesn't exist anymore.

There are 4 possible encodings of derived:

Using a lambda:

inline given derived[T]: (m: Mirror.Of[T]) => Eq[T] = {
    val elemInstances = summonAll[m.MirroredElemTypes]
    inline m match {
      case s: Mirror.SumOf[T]     => eqSum(s, elemInstances)
      case p: Mirror.ProductOf[T] => eqProduct(p, elemInstances)
    }
  }

Using a lambda with a given:

inline given derived[T]: (given m: Mirror.Of[T]) => Eq[T] = {
    val elemInstances = summonAll[m.MirroredElemTypes]
    inline m match {
      case s: Mirror.SumOf[T]     => eqSum(s, elemInstances)
      case p: Mirror.ProductOf[T] => eqProduct(p, elemInstances)
    }
  }

Directly using a given:

inline given derived[T](given m: Mirror.Of[T]): Eq[T] = {
    val elemInstances = summonAll[m.MirroredElemTypes]
    inline m match {
      case s: Mirror.SumOf[T]     => eqSum(s, elemInstances)
      case p: Mirror.ProductOf[T] => eqProduct(p, elemInstances)
    }
  }

Using summonFrom.

  inline given derived[T]: Eq[T] =
    summonFrom {
      case m: Mirror.Of[T] =>
        val elemInstances = summonAll[m.MirroredElemTypes]
        inline m match {
          case s: Mirror.SumOf[T]     => eqSum(s, elemInstances)
          case p: Mirror.ProductOf[T] => eqProduct(p, elemInstances)
        }
    }

I would go as far as to say that there should be a regression test for all four variants and derivation.md should mention the summonFrom approach.

deusaquilus commented 4 years ago

Also, interestingly eqGen above is does not compile when I try inline given instead of inline implicit def. That needs to be investigated before implicits are deprecated.

object Macro_4 {
  implicit inline def eqGen[T]: Eq[T] = Eq.derived
  // inline given autoEq[T]: Eq[T] = Eq.derived // This does not work!!

  inline def [T](x: =>T) === (y: =>T): Boolean = {
    val eq = summon[Eq[T]]
    eq.eqv(x, y)
  }
}

deusaquilus commented 4 years ago

Here's how I would do it: #8087.

scala / scala3