avro-kotlin / avro4k

Avro format support for Kotlin
Apache License 2.0
198 stars 37 forks source link

Support contextual serializers #88

Closed rocketraman closed 3 years ago

rocketraman commented 4 years ago

I can do this:

@Serializable
data class Foo(
  @Contextual
  val d: LocalDate
)

and create a JSON serializer like this:

Json {
    serializersModule = SerializersModule {
        contextual(LocalDateAsLongSerializable)
    }
}

however if I try to do a similar same thing with Avro4k like this:

Avro(SerializersModule {
    contextual(LocalDateSerializer())
})

at runtime I get the error:

Caused by: kotlinx.serialization.SerializationException: Unsupported type kotlinx.serialization.ContextualSerializer<LocalDate> of CONTEXTUAL
    at com.github.avrokotlin.avro4k.schema.SchemaForKt.schemaFor(SchemaFor.kt:180)
    at com.github.avrokotlin.avro4k.schema.ClassSchemaFor.buildField(ClassSchemaFor.kt:78)
    at com.github.avrokotlin.avro4k.schema.ClassSchemaFor.dataClassSchema(ClassSchemaFor.kt:63)
    at com.github.avrokotlin.avro4k.schema.ClassSchemaFor.schema(ClassSchemaFor.kt:43)
    at com.github.avrokotlin.avro4k.Avro.schema(Avro.kt:266)
    at com.github.avrokotlin.avro4k.Avro.schema(Avro.kt:269)
    at com.github.avrokotlin.avro4k.AvroOutputStreamBuilder.to(Avro.kt:121)
jeshall commented 3 years ago

I've encountered the same issue when attempting to write a class that satisfies the spec for the Avro Event Format for CloudEvents - Version 1.0.1

The issue stems from the multiple value types in the attribute field defined as

{
  "name": "attribute",
  "type": {
    "type": "map",
    "values": ["null", "boolean", "int", "string", "bytes"]
   }
}

The following code:

  @Serializable
  class AvroCloudEvent(
    val attribute: Map<String, @Contextual Any?>
  )

  val module = SerializersModule {
    polymorphic(baseClass = Any::class, actualClass = Int::class, actualSerializer = Int.serializer())
    polymorphic(baseClass = Any::class, actualClass = String::class, actualSerializer = String.serializer())
    polymorphic(baseClass = Any::class, actualClass = Boolean::class, actualSerializer = Boolean.serializer())
  }

  val avro = Avro(serializersModule = module)

  println(avro.schema(AvroCloudEvent.serializer()))

Results in the exception:

Exception in thread "main" kotlinx.serialization.SerializationException: Unsupported type kotlinx.serialization.ContextualSerializer<Any>? of CONTEXTUAL
    at com.github.avrokotlin.avro4k.schema.SchemaForKt.schemaFor(SchemaFor.kt:180)
    at com.github.avrokotlin.avro4k.schema.MapSchemaFor.schema(SchemaFor.kt:117)
    at com.github.avrokotlin.avro4k.schema.ClassSchemaFor.buildField(ClassSchemaFor.kt:84)
    at com.github.avrokotlin.avro4k.schema.ClassSchemaFor.dataClassSchema(ClassSchemaFor.kt:63)
    at com.github.avrokotlin.avro4k.schema.ClassSchemaFor.schema(ClassSchemaFor.kt:43)
    at com.github.avrokotlin.avro4k.Avro.schema(Avro.kt:266)
    at com.github.avrokotlin.avro4k.Avro.schema(Avro.kt:269)
geomagilles commented 3 years ago

@rocketraman You can write:

@Serializable
data class Foo(
    @Serializable(with = LocalDateSerializer::class)
    val d: LocalDate
)

and this works

val b = Avro.default.encodeToByteArray(Foo.serializer(), Foo(LocalDate.now()))
val c = Avro.default.decodeFromByteArray(Foo.serializer(), b)

LocalDateSerializer implementation in com.github.avrokotlin.avro4k.serializer

vladimirfx commented 3 years ago

In our project, we use 3 serialization formats (JSON, Avro, Protobuf) and Kotlin Serialization is the only solution at present that gives the ability to use one set of model classes for different serialization forms. But effective implementation is possible only with contextual serializers. Please review my MR for this.

jeshall commented 3 years ago

@vladimirfx, it looks the test you added is failing.

https://github.com/avro-kotlin/avro4k/pull/99/checks

vladimirfx commented 3 years ago

@vladimirfx, it looks the test you added is failing.

https://github.com/avro-kotlin/avro4k/pull/99/checks

Yes, it is stated in PR #99 that it depends on contextual serialization. Merge contextual serialization PR #98 first and then the second PR #99 will work.