WIP: Schema mapping using isomorphisms

[jdegoes]

It'd be nice that if you had a Schema[A], you could imap that to a Schema[B] by providing Iso[A, B]. I don't know if that's possible but I think it should be, by going through and using Iso on all the lenses / prisms / traversals.

[jkobejs]

I started thinking about solution and at first I thought that it will be simple to implement it, that I just need to implement invariant functor and use Iso[A, B] on schemas lenses, prisms and traversals as you said but I hit stumbling block.

Here is an example. Lets imagine that we have person case class:

case class Person(name: String, age: Int)

We know that person is isomorphic to tuple (String, Int) but it is also isomorphic to tuple (List[Char], Int). If we want to convert Person schema to (String, Int) schema we are ok, but problem arises when we want it to convert to (List[Char], Int) schema.

It is easy to come up with Iso[Person, (List[Char], Int) using functions:

def f(p: Person): (List[Char], Int) = (p.name.toList, p.age)
def g(t: (List[Char], Int)): Person = Person(t._1.mkString, t._2)

val personToTupleIso: Iso[Person, (List[Char], Int)] = Iso(f)(g)

Now lets imagine that we have schema for Person, it is record schema that looks something like this:

val nameLens = Lens[Person, String](_.name)(n => p => p.copy(name = n))
val ageLens = Lens[Person, Int](_.age)(a => p => p.copy(age = a))

val personSchema: Schema[Person] = record[Person](
  ^(
      essentialField[Person, String](
        "name",
         prim(JsonString),
         nameLens
       ),
       essentialField[Person, Int](
         "age",
          prim(JsonNumber),
          ageLens
       )
     )(Person.apply)
  )

We need to get Schema[Tuple[List[Char], Int] using personSchema and personToTupleIso. For that we need somehow convert nameLens and ageLens to lenses with types that look like:

val firstElementLens: Lens[Tuple[List[Char], Int], List[Char] = ...
val secondElementLens: Lens[Tuple[List[Char], Int] = ...

secondElementLens is easy to create:

val secondElementLens: Lens[(List[Char], Int), Int] = personToTupleIso.reverse composeLens ageLens

But to create firstElementLens we need more information since only lens that we can create is:

val lens: Lens[(List[Char], Int), String] =  personToTupleIso.reverse composeLens nameLens

I know that String is isomorphic to List[Char] and I know that it is encoded inside personToTupleIso but I cannot extract it out to use it here. It would be great if I could somehow extract Iso[String, List[Char]] out of Iso[Person, (List[Char], Int)] but I don’t see or know how to do that. @jdegoes and @vil1 do you have suggestions how to do that?

I looked at xenomorph to see how Kris Nuttycombe implemented iso. He has defined new shema for iso called IsoSchema that contains source schema and iso, it looks something like this:

case class IsoSchema(base: F[I], iso: Iso[I, J])

and then when he wants to interpret it, for example in ToGen module, he creates Gen out of source/base schema and then uses iso to map Gen to wanted type.

[vil1]

@josipgrgurica for the purpose of this issue, I think having an IsoSchema is the way to go. For starters, it would allow for easy "map fusion" on schemas (if you map a schema multiple times, you can fuse the various isomorphisms into a single IsoSchema).

FTR, my intuition here is that we cannot settle for a regular map using simple A => B functions because, depending on the feature we implement, Schema will act as a covariant functor (like when producing a Gen[A]) and some times as a contravariant one (like when producing deserializers). So we're bound to use Iso (that go both ways) instead of simple functions (that only go one way).

In summary, for the purpose of this issue, we only want to be able to plug an Iso onto a Schema "from the outside". That is: everything we can do with a Schema[A], we want ot be able to do with a Schema[B], as long as we have an Iso[A, B].

Regarding the other part of your question (basically, plugging an Iso inside some part of a Schema to allow for different representations of the same data type), I do think it's a desirable feature too. Some users might need to say "this specific field is an X in my ADT, but I need to treat it as an Y when dealing with this format Z". This could be partially addressed by the IsoSchema you suggest: the user would be able to state, when defining a schema, that some field must be treated specifically (by plugging some Iso after the "natural" Lens/Prism for that field). Of course this wouldn't address the whole problem. Given an already created schema, we still wouldn't be able to to customise the way data at a specific path is represented in a specific context (eg. String vs List[Char]), but I suppose this should be the concern of another issue.

So in conclusion, the IsoSchema solution is worth giving a try.

[jkobejs]

@vil1 Ok, I can then take this issue after I finish Gen module. I think that your intuition why we need Iso here is good, mine is the same :smile:

[vil1]

FTR solving this issue is needed by #10.

[jkobejs]

I'll take this one.

[jdegoes]

final case class SchemaIso[A0, A](base: Schema[A0], iso: Iso[A0, A]) extends Schema[A]

Then add imap to Schema:

def imap[B](iso: Iso[A, B]): Schema[B] = SchemaIso(self, iso)