godatadriven / pydantic-avro

This library can convert a pydantic class to a avro schema or generate python code from a avro schema.
https://github.com/godatadriven/pydantic-avro
MIT License
63 stars 30 forks source link

Schemas with namespaces break on reuse #61

Closed anthonymckale-6point6 closed 1 month ago

anthonymckale-6point6 commented 1 year ago

When java avro-tools convert .avdl files into .avsc files

the files are created with namespaces which aren't supported by pydantic-avro

https://avro.apache.org/docs/1.10.2/spec.html

this breaks classes when they are reused, as the key doesn't match

aka ".Class" is stored in classes as "Class" instead of ".Class"

written a small change when looking up a reused class will look at the Class portion of namespaced types

example

{
            "name": "Test",
            "type": "record",
            "fields": [
                {
                    "name": "col1",
                    "type": {
                        "type": "record",
                        "name": "Nested",
                        "namespace": "com.pydantic",
                        "fields": [{"name": "col1", "type": "string"}]},
                },
                {
                    "name": "col2",
                    "type": "com.pydantic.Nested",
                },
            ],
        }

Note existing edge case where there is a clash with namespaces but that already exists for non reused case