carreraGroup / json-to-avro-schema

Converts JSON schema to AVRO schema
Apache License 2.0
2 stars 1 forks source link

definitions #16

Closed rubberduck203 closed 3 years ago

rubberduck203 commented 3 years ago

JSON Schema uses this to define types that can be referenced from elsewhere in the schema. https://tools.ietf.org/html/draft-wright-json-schema-validation-01#section-7.1

Closely related to #3

rubberduck203 commented 3 years ago

Avro allows you to reference previously defined records by full name. The first time it is reference, it is defined inline, then each successive time it is referenced it can be referenced by using a fully qualified name.

{
  "type": "record",
  "name": "Definitions",
  "namespace": "com.example",
  "fields": [
    {
      "name": "optimize",
      "type": {
        "name": "someBool",
        "type": "record",
        "fields": [
          {
            "name": "value",
            "type": "boolean"
          }
        ]
      }
    },
    {
      "name": "warnings",
      "type": "someBool"
    }
  ]
}

This introduces some interesting challenges.

Now we must track whether or not we've already defined the definition record.

Also, when transpiling a property, it's key becomes the name of the field. For a definition, the key is the name of the record, so we need to find a different name for the field. (value perhaps?)

rubberduck203 commented 3 years ago

I was curious if we could just inline it all the time. Since we're generating the schema anywhere, there wouldn't be any maintenance burden to duplicating the subschemas. Unfortunately, it doesn't work.


{
  "type": "record",
  "name": "schema",
  "namespace": "com.example",
  "fields": [
    {
      "name": "applesauce",
      "type": {
        "name": "foo",
        "type": "record",
        "fields": [
            { "name": "value", "type": "boolean"}
        ]
      }
    },
    {
      "name": "oj",
      "type": {
        "name": "foo",
        "type": "record",
        "fields": [
            { "name": "value", "type": "boolean"}
        ]
      }
    }
  ]
}
rubberduck@Christophers-MacBook-Pro:~/src/temp $ avro-tools compile schema test2.avsc .
Input files to compile:
  test2.avsc
Exception in thread "main" org.apache.avro.SchemaParseException: Can't redefine: com.example.foo
    at org.apache.avro.Schema$Names.put(Schema.java:1547)
    at org.apache.avro.Schema$Names.add(Schema.java:1541)
    at org.apache.avro.Schema.parse(Schema.java:1660)
    at org.apache.avro.Schema.parse(Schema.java:1673)
    at org.apache.avro.Schema$Parser.parse(Schema.java:1430)
    at org.apache.avro.Schema$Parser.parse(Schema.java:1393)
    at org.apache.avro.tool.SpecificCompilerTool.run(SpecificCompilerTool.java:154)
    at org.apache.avro.tool.Main.run(Main.java:67)
    at org.apache.avro.tool.Main.main(Main.java:56)
rubberduck203 commented 3 years ago
rubberduck203 commented 3 years ago