Error with Map<String, ByteArray>

geomagilles commented 3 years ago

This class:

@Serializable
data class Test(
    val meta: Map<String, ByteArray>
)

triggers this error when decoding:

Exception in thread "main" java.lang.ClassCastException: java.nio.HeapByteBuffer cannot be cast to org.apache.avro.generic.GenericArray
    at com.sksamuel.avro4k.decoder.MapDecoder.beginStructure(MapDecoder.kt:100)
    at kotlinx.serialization.internal.AbstractCollectionSerializer.merge(CollectionSerializers.kt:29)
    at kotlinx.serialization.internal.PrimitiveArraySerializer.deserialize(CollectionSerializers.kt:179)
    at kotlinx.serialization.encoding.Decoder$DefaultImpls.decodeSerializableValue(Decoding.kt:234)
    at kotlinx.serialization.encoding.AbstractDecoder.decodeSerializableValue(AbstractDecoder.kt:17)
    at kotlinx.serialization.encoding.AbstractDecoder.decodeSerializableValue(AbstractDecoder.kt:41)
    at kotlinx.serialization.encoding.AbstractDecoder.decodeSerializableElement(AbstractDecoder.kt:63)
    at kotlinx.serialization.encoding.CompositeDecoder$DefaultImpls.decodeSerializableElement$default(Decoding.kt:479)
    at kotlinx.serialization.internal.MapLikeSerializer.readElement(CollectionSerializers.kt:111)
    at kotlinx.serialization.internal.MapLikeSerializer.readElement(CollectionSerializers.kt:85)
    at kotlinx.serialization.internal.AbstractCollectionSerializer.readElement$default(CollectionSerializers.kt:51)
    at kotlinx.serialization.internal.AbstractCollectionSerializer.merge(CollectionSerializers.kt:36)
    at kotlinx.serialization.internal.AbstractCollectionSerializer.deserialize(CollectionSerializers.kt:43)
    at kotlinx.serialization.encoding.Decoder$DefaultImpls.decodeSerializableValue(Decoding.kt:234)
    at kotlinx.serialization.encoding.AbstractDecoder.decodeSerializableValue(AbstractDecoder.kt:17)
    at kotlinx.serialization.encoding.AbstractDecoder.decodeSerializableValue(AbstractDecoder.kt:41)
    at kotlinx.serialization.encoding.AbstractDecoder.decodeSerializableElement(AbstractDecoder.kt:63)
    at io.infinitic.common.Test$$serializer.deserialize(main.kt)
    at io.infinitic.common.Test$$serializer.deserialize(main.kt:46)
    at kotlinx.serialization.encoding.Decoder$DefaultImpls.decodeSerializableValue(Decoding.kt:234)
    at kotlinx.serialization.encoding.AbstractDecoder.decodeSerializableValue(AbstractDecoder.kt:17)
    at com.sksamuel.avro4k.Avro.fromRecord(Avro.kt:230)
    at com.sksamuel.avro4k.Avro$openInputStream$builder$2.invoke(Avro.kt:165)
    at com.sksamuel.avro4k.io.AvroDataInputStream.next(AvroDataInputStream.kt:27)
    at com.sksamuel.avro4k.io.AvroInputStream$DefaultImpls.nextOrThrow(AvroInputStream.kt:37)
    at com.sksamuel.avro4k.io.AvroDataInputStream.nextOrThrow(AvroDataInputStream.kt:9)
    at com.sksamuel.avro4k.Avro.decodeFromByteArray(Avro.kt:127)

For example with

val o = Test(mapOf("foo" to "bar".toByteArray()))
val serializer = Test.serializer()
val byteArray =  Avro.default.encodeToByteArray(serializer, o)
val o2 = Avro.default.decodeFromByteArray(serializer, byteArray)

While same Json ser/de works and despite a correct avro schema generated:|

{
  "type" : "record",
  "name" : "Test",
  "namespace" : "me",
  "fields" : [ {
    "name" : "meta",
    "type" : {
      "type" : "map",
      "values" : "bytes"
    }
  } ]
}

daicode-jan commented 3 years ago

Hi As far as I know, there is no decoder at the moment that handles byteArray.

For now, you could use something similar like this:


   "type":"record",
   "name":"Test",
   "namespace":"com.example",
   "fields":[
      {
         "name":"payload",
         "type":{
            "type":"map",
            "values":{
               "type":"record",
               "name":"ByteArrayHolder",
               "fields":[
                  {
                     "name":"bytes",
                     "type":"bytes"
                  }
               ]
            }
         }
      }
   ]
}

geomagilles commented 3 years ago

As far as I know, there is no decoder at the moment that handles byteArray.

I believe this is ByteArraySerializer()

geomagilles commented 3 years ago

PR: https://github.com/avro-kotlin/avro4k/pull/79

avro-kotlin / avro4k

Error with Map<String, ByteArray> #76