Support data class optional fields without @AvroDefault

[PhilippZugColenet]

Hello,

when I try to deserialize an Avro message from a JSON that is missing an optional field - with default null - the I got an AvroTypeException.

Used Avro4k 1.10.1

Given the following Avro Schema:

  "type" : "record",
  "name" : "ExampleData",
  "namespace" : "test",
  "fields" : [ {
    "name" : "mandatoryString",
    "type" : "string"
  }, {
    "name" : "optionalString",
    "type" : [ "null", "string" ],
    "default" : null
  }, {
    "name" : "mandatoryInt",
    "type" : "int"
  } ]
}

the following data.json:

{
  "mandatoryString" : "A",
  "mandatoryInt" : 1
}

and the following data class

@Serializable
data class ExampleData(
    val mandatoryString: String,
    val optionalString: String? = null,
    val mandatoryInt: Int
)

I got this exception: org.apache.avro.AvroTypeException: Expected field name not found: optionalString.

Kotlin Test-Code to reproduce (data.json contains the data like above):

@Test
fun `Very basic de-serial`() {
    val avroMapper = Avro(
        configuration = AvroConfiguration(implicitNulls = true)
    )
    @Serializable
    data class ExampleData(val mandatoryString: String, val optionalString: String? = null, val mandatoryInt: Int)

    val exampleDataSchema = avroMapper.schema(ExampleData.serializer())
    println(exampleDataSchema.toString(true))

    val input = avroMapper.openInputStream(ExampleData.serializer()) {
        decodeFormat = AvroDecodeFormat.Json(exampleDataSchema)
    }.from(File("data.json"))
    input.iterator().forEach { println(it) }
    input.close()
}

I my opinion this is a bug and I would expect it to accept a message where an optional field is missing since it has a default value.

Or did I do something wrong? Can you help me please?

[Chuckame]

Hey,

Thanks for reaching us about this bug. I've just checked it, and surprisingly it fails. I'm currently trying to fix it!

[Chuckame]

It's more complex than expected. The main issue is that Avro4k (especially kotlinx-serialization) is not able to retrieve the default value for optional kotlin fields, that's why it has been created an @AvroDefault annotation.

To fix it, I would need to use the writer schema for reading, then adapting the writer schema to the reader schema/data class. This could be ok to do, but requires a lot of code.

Currently, a v2 of avro4k is in progress fixing a lot of stuff, improving performances and the Avro entrypoint has been totally revamped for better and simpler usability.

I will take this feature into account. Until that moment, please provide the default value through the @AvroDefault. Sorry for this workaround, and thanks for your comprehension!