avro-kotlin / avro4k

Avro format support for Kotlin
Apache License 2.0
188 stars 36 forks source link

Question: How to convert from GenericRecord? #139

Closed jangalinski closed 1 year ago

jangalinski commented 1 year ago

Sorry for spamming you with questions, if there is a better way to contact you and discuss features, please let me know.

I did a first work around for dealing with single object encoding: I decode the byte buffer to a GenericRecord like this:

val record: GenericData.Record = BinaryMessageDecoder<GenericData.Record>(
      GenericData().apply {
        addLogicalTypeConversion(Conversions.UUIDConversion())
        addLogicalTypeConversion(Conversions.DecimalConversion())
      },
      readerSchema
    ).apply {
      addSchema(writerSchema)
    }.decode(singleObjectEncodedByteBuffer)

This gives me a record containing my message data in avro-java specific types (Utf8 for string, ...)

I was hoping I could use your Avro.default.fromRecord() util to convert the record into a type safe data class, using your existing serializers. This would not be an ideal solution, because of the two-step deserialization, but could be a first step towards single object support.

However, the fromRecordfunction requires me to pass a DeserializationStrategy and I do not understand what use here, as all implementations seem to be very field specific, while my record is a composite containing all values of the message (uuid-id, instant-timestamp, enum-type, string(utf8-message) ...

Is my usecase supported?

Thanks again.

thake commented 1 year ago

No worries, as we don't use github discussions at the moment, asking questions in issues is the way to go.

Did you have a look at the tests in this project? All intended use cases can be found there. Here is the test that handles reading generic records: https://github.com/avro-kotlin/avro4k/blob/f6a7c0c03a41eddb15853cb595585d3f1fd337bd/src/test/kotlin/com/github/avrokotlin/avro4k/decoder/BasicDecoderTest.kt#L32-L36

Performance-wise, it doesn't make a difference if you use the java avro library before to create a GenericRecord and then handle it over to avro4k. For the supported encodings it is done in the same way by this library. First, the java avro library Decoders decodes the byte array to a GenericRecord. Afterward, that GenericRecord will be mapped to the kotlin data structure.

I hope this helps you. If you have any further questions, please don't hesitate to ask.

jangalinski commented 1 year ago

Ah, cool, yes that worked for me ... I was a bit confused because the examples used in tests where named "float", "string", ... but it also worked with a complex data class containing enums, uuid, decimal, nullables ....

very nice.

I will close this one