My coworker @maxstreese recently tried backward compatibility using avro4s, but it didn't work out. Here's his code:
import com.sksamuel.avro4s.*
import java.io.*
@AvroDoc("initial")
@AvroName("Data")
case class Initial(a: Int, b: String)
@AvroDoc("updated")
@AvroName("Data")
case class Updated(a: Int, c: String = "foo")
@main
def main(): Unit =
val buffer = ByteArrayOutputStream()
// producer
val out = AvroOutputStream.binary[Initial].to(buffer).build()
out.write(Initial(1, "x"))
out.close()
// consumer
val in = new AvroBinaryInputStream[Updated](ByteArrayInputStream(buffer.toByteArray), AvroSchema[Initial])
println(in.iterator.next)
Clearly this should work because removed fields don't affect backward compatibility and there is a default for the new field c. But it doesn't work because AvroBinaryInputStream doesn't supply the reader schema to the decoder.
This PR adds a new constructor for AvroBinaryInputStream that allows supplying a reader schema.
I also got rid of one of those ugly asInstanceOf calls while I was at it.
Here are some related issues that this PR doesn't fix:
I don't know how to fix the helper methods in the AvroInputStream object because that would break binary compatibility
AvroDataInputStream is probably suffering the same issue
Hey there,
My coworker @maxstreese recently tried backward compatibility using avro4s, but it didn't work out. Here's his code:
Clearly this should work because removed fields don't affect backward compatibility and there is a default for the new field
c
. But it doesn't work becauseAvroBinaryInputStream
doesn't supply the reader schema to the decoder.This PR adds a new constructor for
AvroBinaryInputStream
that allows supplying a reader schema. I also got rid of one of those uglyasInstanceOf
calls while I was at it.Here are some related issues that this PR doesn't fix:
AvroInputStream
object because that would break binary compatibilityAvroDataInputStream
is probably suffering the same issue