search

astronomer / astro-sdk

Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
https://astro-sdk-python.rtfd.io/
Apache License 2.0
343 stars 42 forks source link

Limit Pydantic < 2.0.0 until Airflow resolves incompatibilities #1981

Closed tatiana closed 1 year ago

tatiana commented 1 year ago

Before this change, the project's main branch tests failed.

Example of error:

    module = import_module(module_path)
  File "/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/runner/work/astro-sdk/astro-sdk/python-sdk/.nox/test_examples_by_dependency-extras-sqlite-include-sqlite/lib/python3.8/site-packages/airflow/models/skipmixin.py", line 25, in <module>
    from airflow.serialization.pydantic.dag_run import DagRunPydantic
  File "/home/runner/work/astro-sdk/astro-sdk/python-sdk/.nox/test_examples_by_dependency-extras-sqlite-include-sqlite/lib/python3.8/site-packages/airflow/serialization/pydantic/dag_run.py", line 24, in <module>
    from airflow.serialization.pydantic.dataset import DatasetEventPydantic
  File "/home/runner/work/astro-sdk/astro-sdk/python-sdk/.nox/test_examples_by_dependency-extras-sqlite-include-sqlite/lib/python3.8/site-packages/airflow/serialization/pydantic/dataset.py", line 40, in <module>
    class TaskOutletDatasetReferencePydantic(BaseModelPydantic):
  File "/home/runner/work/astro-sdk/astro-sdk/python-sdk/.nox/test_examples_by_dependency-extras-sqlite-include-sqlite/lib/python3.8/site-packages/pydantic/_internal/_model_construction.py", line 95, in __new__
    private_attributes = inspect_namespace(
  File "/home/runner/work/astro-sdk/astro-sdk/python-sdk/.nox/test_examples_by_dependency-extras-sqlite-include-sqlite/lib/python3.8/site-packages/pydantic/_internal/_model_construction.py", line 328, in inspect_namespace
    raise PydanticUserError(
pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = <class 'str'>`. All model fields require a type annotation; if `dag_id` is not meant to be a field, you may be able to resolve this error by annotating it as a `ClassVar` or updating `model_config['ignored_types']`.

From: https://github.com/astronomer/astro-sdk/actions/runs/5449375269/jobs/9913528880

Relates to: https://github.com/apache/airflow/issues/32311

tatiana commented 1 year ago

Ourmaster branch tests are currently broken due to recent permission changes between the account used in the CI and the S3 bucket used for testing. The current broken test relates to this.