beam-community / avro_ex

An Avro Library that emphasizes testability and ease of use.
https://hexdocs.pm/avro_ex/AvroEx.html
67 stars 27 forks source link

default values of `null` are removed from schema in `encode_schema` #77

Open otosky opened 2 years ago

otosky commented 2 years ago

Heyo 👋

Ran into an edge case today where a schema defined in JSON containing nullable unions would have the "default": null portion stripped out if it was run through a sequence of AvroEx.decode_schema -> AvroEx.encode_schema.

Example:

schema = AvroEx.decode_schema!(~S({
  "type": "record", 
  "name": "Record", 
  "fields": [
     {"type": ["null", "int"], "name": "nullable", "default": null}
   ]
}))
AvroEx.encode_schema(schema)

Output:

"{\"fields\":[{\"name\":\"nullable\",\"type\":[{\"type\":\"null\"},{\"type\":\"int\"}]}],\"name\":\"Record\",\"type\":\"record\"}"

I haven't dug too deep, but other primitives seem okay:

schema = AvroEx.decode_schema!(~S({
  "type": "record", 
  "name": "Record", 
  "fields": [
     {"type": ["null", "int"], "name": "nullable", "default": 5}
   ]
}))
AvroEx.encode_schema(schema)
"{\"fields\":[{\"default\":5,\"name\":\"nullable\",\"type\":[{\"type\":\"null\"},{\"type\":\"int\"}]}],\"name\":\"Record\",\"type\":\"record\"}"

Using v2.0.1

davydog187 commented 2 years ago

Thanks for the report @otosky :)

davydog187 commented 2 years ago

@marpo60 told me he is working on a fix