sksamuel / avro4s

Avro schema generation and serialization / deserialization for Scala
Apache License 2.0
719 stars 237 forks source link

use correct reader schema in AvroBinaryInputStream #817

Closed mberndt123 closed 8 months ago

mberndt123 commented 8 months ago

Hey there,

My coworker @maxstreese recently tried backward compatibility using avro4s, but it didn't work out. Here's his code:

import com.sksamuel.avro4s.*
import java.io.*

@AvroDoc("initial")
@AvroName("Data")
case class Initial(a: Int, b: String)

@AvroDoc("updated")
@AvroName("Data")
case class Updated(a: Int, c: String = "foo")

@main
def main(): Unit =
  val buffer = ByteArrayOutputStream()

  // producer
  val out = AvroOutputStream.binary[Initial].to(buffer).build()
  out.write(Initial(1, "x"))
  out.close()

  // consumer
  val in = new AvroBinaryInputStream[Updated](ByteArrayInputStream(buffer.toByteArray), AvroSchema[Initial])
  println(in.iterator.next)

Clearly this should work because removed fields don't affect backward compatibility and there is a default for the new field c. But it doesn't work because AvroBinaryInputStream doesn't supply the reader schema to the decoder.

This PR adds a new constructor for AvroBinaryInputStream that allows supplying a reader schema. I also got rid of one of those ugly asInstanceOf calls while I was at it.

Here are some related issues that this PR doesn't fix:

sksamuel commented 8 months ago

Nice!