picklers could be contravariant?

[havocp]

I'm just initially investigating the library (we are still trying to figure out how to serialize stuff in sbt).

One thing we discovered using play-json is that we ended up going back and changing a lot of Format[T] to Writes[T], where Writes is contravariant in T. This then enables correct variance (i.e. not invariance) on types which make use of Writes[T]. Format[T] forced invariance in unnatural places. A simple example is JsonSink[-T] here https://github.com/sbt/sbt-remote-control/blob/4d9b4dbbff8684dff13406bcd974f94f830f8de4/server/src/main/scala/sbt/server/ServerListener.scala#L9

trait JsonSink[-J] {
  def send[T <: J: Writes](msg: T): Unit
}

So for example we have JsonSink[Event] which can be used where JsonSink[LogEvent] is expected, assuming LogEvent extends Event.

The way play-json works is that people are only really supposed to explicitly write implicit Reads and Writes. Then the invariant Format gets auto-generated https://github.com/playframework/playframework/blob/master/framework/src/play-json/src/main/scala/play/api/libs/json/Format.scala#L75

 implicit def GenericFormat[T](implicit fjs: Reads[T], tjs: Writes[T]): Format[T] = {
  new Format[T] {
    def reads(json: JsValue) = fjs.reads(json)
    def writes(o: T) = tjs.writes(o)
  }
}

Play has a macro Json.format[T] to generate an entire Format, but we found this was kind of a pitfall (it shouldn't be used, only Json.writes[T] and Json.reads[T] should be - the Format gets auto-defined by the GenericFormat implicit above).

Because Format extends both Reads and Writes, if you define an implicit Format explicitly it even breaks things (if both a Reads and a Format are available it's ambiguous when someone asks for Reads).

Strictly speaking Format isn't needed in play-json - you could always replace (implicit format: Format[T]) with (implicit reads: Reads[T], writes: Writes[T]) - so Format is just sugar.

It looks like the incidental reason SPickler is invariant is that it has "val format" but perhaps that can be changed.

It is fairly normal to have "unidirectional" serializations I think; client and server may each have things they know about which they send but don't receive, or vice versa. For example in sbt-remote-control only the server sends Event instances.

cc @jsuereth

[havocp]

Realized what I said about "val format" is wrong because PickleFormat has no type parameter.

[phaller]

I've created https://github.com/scala/pickling/pull/192 which investigates this. It makes SPickler contravariant. Currently, only two tests break, which pickle abstract IndexedSeq[Int].

Support for pickling abstract collections has always been ad-hoc. Thus, defining robust rules for abstract types could enable at the same time (a) contravariant picklers, and (b) a more principled story for pickling collections. Maybe limiting to pickling only concrete collection types would be sufficient?

[havocp]

Thanks for looking into it, Philipp.

[phaller]

Based on many subsequent discussions, this issue is clearly outdated. Therefore, I'm closing it now.