actgardner / gogen-avro

Generate Go code to serialize and deserialize Avro schemas
MIT License
366 stars 87 forks source link

Issue with schema evolution when adding an enum #185

Closed bozhidarG closed 2 years ago

bozhidarG commented 2 years ago

Hello i've stumbled upon an issue in the latest version v10.2.0 , i've also managed to reproduce it with some simple schemas. We have this issue only when we are serializing with one schema and deserializing with a backward compatible one. The schema used to serialize the msg is as follows

{
  "type": "record",
  "name": "TestEnumSchemaEvolution",
  "namespace": "test.schema.evolution",
  "fields": [
    {
      "name": "id",
      "type": "string"
    },
    {
      "name": "added_enum",
      "type":
      [
        "null",
        {
          "type": "enum",
          "name": "SchemaAddedEnum",
          "symbols":
          [
            "enum1",
            "enum2",
            "unknown"
          ],
          "default": "unknown"
        }
      ],
      "default": null
    }
  ]
}

The schema used to deserialize :

{
  "type": "record",
  "name": "TestEnumSchemaEvolution",
  "namespace": "test.schema.evolution",
  "fields": [
    {
      "name": "id",
      "type": "string"
    }
  ]
}

For both schemas i've used the cli tool to generate static code and use the structs. v10 of the tool. The simple test code is this one:

enumPayload := writer_simple.NewTestEnumSchemaEvolution()
enumPayload.Id = "test-id"
enumPayload.Added_enum = writer_simple.NewUnionNullSchemaAddedEnum()
enumPayload.Added_enum.SchemaAddedEnum = writer_simple.SchemaAddedEnumEnum2
enumPayload.Added_enum.UnionType = writer_simple.UnionNullSchemaAddedEnumTypeEnumSchemaAddedEnum

avroPayload := bytes.Buffer{}
err := enumPayload.Serialize(&avroPayload)
fmt.Println(err)

onlyIdPayload := reader_simple.NewTestEnumSchemaEvolution()
program, err := compiler.CompileSchemaBytes([]byte(enumPayload.Schema()), []byte(onlyIdPayload.Schema()))
fmt.Println(err)

buf := bytes.NewBuffer(avroPayload.Bytes())

errn := vm.Eval(buf, program, &onlyIdPayload)
fmt.Println(errn)
jbytes, _ := json.Marshal(&onlyIdPayload)
fmt.Println(string(jbytes))

The error i get is on the vm.Eval method and is as follows:

Runtime error: Unexpected value for enum, frame: {false 0 1 0 0 []  true}, pc: 0xc0000222d8

I've tested the same thing using v9.2.0 and there were no issues there. Also might be something i'm doing wrong, would be very grateful if you can help with some info on this one. I'll be using v9 for now.

actgardner commented 2 years ago

Hi @bozhidarG! Thanks for such a clear error report. I was able to fix this issue in the most recent release (10.2.1). Let me know if it works for you.

bozhidarG commented 2 years ago

Thanks, all works as expected now.