jpmorganchase / py-avro-schema

Generate Apache Avro schemas for Python types including standard library data-classes and Pydantic data models.
https://py-avro-schema.readthedocs.io/
Apache License 2.0
37 stars 6 forks source link

Using generics as types does not work anymore #81

Open damey2011 opened 3 days ago

damey2011 commented 3 days ago

Example code snippet

class EntityA(BaseModel):
    name: str
    age: int

class EntityB(BaseModel):
    occupation: str
    salary: float

EntityType = TypeVar("EntityType", bound=BaseModel)

class ConsumerMessage(Generic[EntityType]):
    action: Literal["create", "update", "delete"]
    entity: EntityType

Generating the avro schema from this used to work fine in 3.0.3 but started to fail when I updated to 3.8.2. I get the following error:

Traceback (most recent call last):
      File "/Users/me/Desktop/app/pubsub-infrastructure/__main__.py", line 3, in <module>
        main()
      File "/Users/me/Desktop/app/pubsub-infrastructure/iac/main.py", line 107, in main
        setup_topics()
      File "/Users/me/Desktop/app/pubsub-infrastructure/iac/main.py", line 65, in setup_topics
        avro_schema = pas.generate(
                      ^^^^^^^^^^^^^
      File "/Users/me/Library/Caches/pypoetry/virtualenvs/app-pubsub-BvA2HnB7-py3.11/lib/python3.11/site-packages/memoization/caching/plain_cache.py", line 42, in wrapper
        result = user_function(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/me/Library/Caches/pypoetry/virtualenvs/app-pubsub-BvA2HnB7-py3.11/lib/python3.11/site-packages/py_avro_schema/__init__.py", line 64, in generate
        schema_dict = schema(py_type, namespace=namespace, options=options)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/me/Library/Caches/pypoetry/virtualenvs/app-pubsub-BvA2HnB7-py3.11/lib/python3.11/site-packages/py_avro_schema/_schemas.py", line 147, in schema
        schema_obj = _schema_obj(py_type, namespace=namespace, options=options)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/me/Library/Caches/pypoetry/virtualenvs/app-pubsub-BvA2HnB7-py3.11/lib/python3.11/site-packages/py_avro_schema/_schemas.py", line 168, in _schema_obj
        schema_obj = schema_class(py_type, namespace=namespace, options=options)  # type: ignore
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/me/Library/Caches/pypoetry/virtualenvs/app-pubsub-BvA2HnB7-py3.11/lib/python3.11/site-packages/py_avro_schema/_schemas.py", line 974, in __init__
        self.record_fields = [self._record_field(name, field) for name, field in self.py_fields.items()]
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/me/Library/Caches/pypoetry/virtualenvs/app-pubsub-BvA2HnB7-py3.11/lib/python3.11/site-packages/py_avro_schema/_schemas.py", line 974, in <listcomp>
        self.record_fields = [self._record_field(name, field) for name, field in self.py_fields.items()]
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/me/Library/Caches/pypoetry/virtualenvs/app-pubsub-BvA2HnB7-py3.11/lib/python3.11/site-packages/py_avro_schema/_schemas.py", line 981, in _record_field
        field_obj = RecordField(
                    ^^^^^^^^^^^^
      File "/Users/me/Library/Caches/pypoetry/virtualenvs/app-pubsub-BvA2HnB7-py3.11/lib/python3.11/site-packages/py_avro_schema/_schemas.py", line 888, in __init__
        self.schema = _schema_obj(self.py_type, namespace=self._namespace, options=options)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/me/Library/Caches/pypoetry/virtualenvs/app-pubsub-BvA2HnB7-py3.11/lib/python3.11/site-packages/py_avro_schema/_schemas.py", line 171, in _schema_obj
        raise TypeNotSupportedError(f"Cannot generate Avro schema for Python type {py_type}")
    py_avro_schema._schemas.TypeNotSupportedError: Cannot generate Avro schema for Python type ~EntityType
faph commented 3 days ago

@damey2011 would you be able to define a unit test to reproduce this issue?