mtth / avsc

Avro for JavaScript :zap:
MIT License
1.27k stars 147 forks source link

IDL not exporting types for array of union #438

Closed johnykov closed 1 year ago

johnykov commented 1 year ago

Hi, I appreciate your hard work on this library and find it helpful! Great job, thank you. I don't know if this is on purpose or just lacking functionality but when converting IDL to Avro Json notation I've noticed that I'm missing some types. They get lost somewhere along the way. For example, having IDL Avro file:

@namespace("com.org.domain.fixtures")
protocol ImportProto {
    record Single {
        string foo;
    }

    record Double {
        string bar;
    }

    record Import {
        array<union{Double, Single}> imports;
    }
}

I get following Avro Schema in the result:

{
  "namespace": "com.org.domain.fixtures",
  "type": "record",
  "name": "Import",
  "fields": [
    {
      "type": {
        "type": "array",
        "items": [
          "Double",
          "Single"
        ]
      },
      "name": "imports"
    }
  ]
}

which obviously is not a valid Avro Schema as any tool reading it throws an error of unrecognized types "Double" and "Single".

I would expect to get valid Avro schema which is:

{
  "namespace": "com.org.domain.fixtures",
  "type": "record",
  "name": "Import",
  "fields": [
    {
      "type": {
        "type": "array",
        "items": [        {
          "type": "record",
          "name": "Single",
          "fields": [
            {
              "type": "string",
              "name": "foo"
            }
          ]
        },{
          "type": "record",
          "name": "Double",
          "fields": [
            {
              "type": "string",
              "name": "bar"
            }
          ]
        }]
      },
      "name": "imports"
    }
  ]
}
mtth commented 1 year ago

Hi @johnykov. Can you also share your code?

johnykov commented 1 year ago

sure, however I'm using avsc indirectly through @kafkajs/confluent-schema-registry

import path from 'path'
import { avdlToAVSCAsync } from '@kafkajs/confluent-schema-registry'
;(async () => {
  const avroSchema = await avdlToAVSCAsync(
    path.join(__dirname, 'schema/import.avdl')
  )

  console.log(JSON.stringify(avroSchema, null, 2))

})()
johnykov commented 1 year ago

I think I need to re-address the issue to @kafkajs/confluent-schema-registry sorry :)