Codec derivation in Scala 3.5

[olafmaurer]

In scala 3.4, the following worked:

import com.github.plokhotnyuk.jsoniter_scala.core.* import com.github.plokhotnyuk.jsoniter_scala.macros.JsonCodecMaker import utils.tagging.Tags.{@@, tag}

inline given tagJsonValueCodec[V, T]: JsonValueCodec[V @@ T] = new JsonValueCodec[V @@ T]: val codec: JsonValueCodec[V] = JsonCodecMaker.makeCirceLikeSnakeCased override def decodeValue(in: JsonReader, default: V @@ T): V @@ T = tag[T](codec.decodeValue(in, default: V)) override def encodeValue(x: V @@ T, out: JsonWriter): Unit = codec.encodeValue(x, out) override def nullValue: V @@ T = tag [T] (codec.nullValue)

Upgrading to Scala 3.5, this yields the warning

[warn] 7 |inline given tagJsonValueCodec[V, T]: JsonValueCodec[V @@ T] = new JsonValueCodec[V @@ T]: [warn] | ^ [warn] | New anonymous class definition will be duplicated at each inline site

so the code was changed to

[same imports] inline given tagJsonValueCodec[V, T]: JsonValueCodec[V @@ T] with val codec: JsonValueCodec[V] = JsonCodecMaker.makeCirceLikeSnakeCased override def decodeValue(in: JsonReader, default: V @@ T): V @@ T = tag[T](codec.decodeValue(in, default: V)) override def encodeValue(x: V @@ T, out: JsonWriter): Unit = codec.encodeValue(x, out) override def nullValue: V @@ T = tag[T](codec.nullValue)

which does not work, it gives

[error] -- Error: [...]/jsoniterCodecs.scala:8:70 [error] 8 | val codec: JsonValueCodec[V] = JsonCodecMaker.makeCirceLikeSnakeCased [error] | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [error] |No implicit 'com.github.plokhotnyuk.jsoniterscala.core.JsonValueCodec[ >: scala.Nothing <: scala.Any]' defined for 'tagJsonValueCodec.this.V'.

Is this some known issue? Any suggestions?

[olafmaurer]

Relevant tagging implementation:

opaque type Tagged[+V, +Tag] = Any type @@[+V, +Tag] = V & Tagged[V, Tag]

def tag[Tag]: [V] => V => V @@ Tag = [V] => (v: V) => v

[plokhotnyuk]

@olafmaurer Thanks for opening an issue!

I'm going to reproduce it and check what is happening in generated code using given CodecMakerConfig.PrintCodec = new CodecMakerConfig.PrintCodec {}.

[plokhotnyuk]

It is a bug that could be reproduces even in Scala 2 for generic implicit def tagJsonValueCodec[V, T](implicit codec: JsonValueCodec[V]).

Scala 3.5.0+ compiler only warns about the issue for some inline given cases.

To see how many instances of codecs instantiated just need to add println("+1") after val codec ... line.

As a workaround you can convert tagJsonValueCodec to simple def and derive codecs for each tagged type, like here:

//> using dep "com.github.plokhotnyuk.jsoniter-scala::jsoniter-scala-core::2.30.9"
//> using compileOnly.dep "com.github.plokhotnyuk.jsoniter-scala::jsoniter-scala-macros::2.30.9"

import com.github.plokhotnyuk.jsoniter_scala.macros._
import com.github.plokhotnyuk.jsoniter_scala.core._

object Tags {
  opaque type Tagged[+V, +Tag] = Any

  type @@[+V, +Tag] = V & Tagged[V, Tag]

  def tag[Tag]: [V] => V => V @@ Tag = [V] => (v: V) => v
}

object Graph {
  import Tags.{@@, tag}

  def tagJsonValueCodec[V, T](codec: JsonValueCodec[V]): JsonValueCodec[V @@ T] = new JsonValueCodec[V @@ T]:
    println("+1")
    override def decodeValue(in: JsonReader, default: V @@ T): V @@ T = tag[T](codec.decodeValue(in, default: V))
    override def encodeValue(x: V @@ T, out: JsonWriter): Unit = codec.encodeValue(x, out)
    override def nullValue: V @@ T = tag[T](codec.nullValue)

  trait NodeIdTag

  type NodeId = Int @@ NodeIdTag

  case class Node(id: NodeId, name: String)
  case class Edge(node1: NodeId, node2: NodeId)
}

//given CodecMakerConfig.PrintCodec = new CodecMakerConfig.PrintCodec {}
given JsonValueCodec[Graph.NodeId] = Graph.tagJsonValueCodec(JsonCodecMaker.make)

given JsonValueCodec[Graph.Node] = JsonCodecMaker.make
given JsonValueCodec[Graph.Edge] = JsonCodecMaker.make

println(readFromString[Graph.Node]("""{"id":1,"name":"VVV"}"""))
println(readFromString[Graph.Edge]("""{"node1":1,"node2":2}"""))

Also, the same issue of instantiation of redundant codecs happening for values stored in val codec.

To mitigate that you need to generate, store (and reuse) those codecs, so instead of

given JsonValueCodec[Graph.NodeId] = Graph.tagJsonValueCodec(JsonCodecMaker.make)
given JsonValueCodec[<some other tagged int type>] = Graph.tagJsonValueCodec(JsonCodecMaker.make)

use

val intCodec: JsonValueCodec[Int] = JsonCodecMaker.make
given JsonValueCodec[Graph.NodeId] = Graph.tagJsonValueCodec(intCodec)
given JsonValueCodec[<some other tagged int type>] = Graph.tagJsonValueCodec(intCodec)

[olafmaurer]

Thanks for the workaround, it was successfully applied :)

[plokhotnyuk]

@olafmaurer I'm happy to see that it was acceptable for you.

Probably it will be possible to add a new feature to avoid need of custom codec creation by introducing acceptance of implicit conversions available in the scope of make calls, like here for Scala 3:

val nodeCodec: JsonValueCodec[Node] = {
  given Conversion[Int, NodeId] with
    def apply(x: Int): NodeId = tag[NodeIdTag](x)

  given Conversion[NodeId, Int] with
    def apply(x: NodeId): Int = x

  make[Node]
}

or here for Scala 2:

val nodeCodec: JsonValueCodec[Node] = {
  import scala.language.implicitConversions

  implicit def convertToTagged(x: Int): NodeId = tag[NodeIdTag](x)

  implicit def convertFromTagged(x: NodeId): Int = x

  make[Node]
}

[plokhotnyuk]

@olafmaurer @nkgm Reaching you here to point on a generalized workaround proposed in this commit.

You still need to provide codecs for types to be used with tags.

Would it be more acceptable for you?

[nkgm]

Hi @plokhotnyuk, I'm not sure how this relates to my self-type issue?

[plokhotnyuk]

Hi @plokhotnyuk, I'm not sure how this relates to my self-type issue?

It is not related your your issue directly. I just saw your code samples with tagged types.

[plokhotnyuk]

I tried the following script using scala-cli and it works fine with Scala 3.5.0, but need to avoid appearing of intCodec and stringCodec in the scope where codecs for other types (non-tagged) will be generated:

//> using dep "com.github.plokhotnyuk.jsoniter-scala::jsoniter-scala-core::2.30.9"
//> using compileOnly.dep "com.github.plokhotnyuk.jsoniter-scala::jsoniter-scala-macros::2.30.9"
//> using scala 3.5.0

import com.github.plokhotnyuk.jsoniter_scala.core.*
import com.github.plokhotnyuk.jsoniter_scala.macros.JsonCodecMaker.*

object Tags {
  opaque type Tagged[+V, +T] = Any

  type @@[+V, +T] = V & Tagged[V, T]

  def tag[T]: [V] => V => V @@ T = [V] => (v: V) => v

  inline given[V, T](using c: JsonValueCodec[V]): JsonValueCodec[V @@ T] = 
    c.asInstanceOf[JsonValueCodec[V @@ T]]
}

import Tags.*

implicit val intCodec: JsonValueCodec[Int] = make
implicit val stringCodec: JsonValueCodec[String] = make

trait NodeIdTag

type NodeId = Int @@ NodeIdTag

trait NodeNameTag

type NodeName = String @@ NodeNameTag

case class Node(id: NodeId, name: NodeName)

case class Edge(n1: NodeId, n2: NodeId)

println(readFromString("""{"id":1,"name":"VVV"}""")(make[Node]))
println(readFromString("""{"n1":1,"n2":2}""")(make[Edge]))

[nkgm]

It is not related your your issue directly. I just saw your code samples with tagged types.

Ah, so more of an extra point. That has always worked for me, but you just gave me this idea:

inline given [F[_], V, T](using c: F[V]): F[V @@ T] = c.asInstanceOf[F[V @@ T]]

Now Tags can happily live in my Predef library while taking care of its implicits.

Edit: I'm also on Scala 3.5.0

[plokhotnyuk]

inline given [F[_], V, T](using c: F[V]): F[V @@ T] = c.asInstanceOf[F[V @@ T]]

Wow! Such universal solution for all type classes!

[plokhotnyuk]

To avoid anonymous class duplication just need to create a named class and use it like here.

To reduce number of created codec instances during derivation create them manually and share using given without parameters (including implicit ones) or implicit val like here.