apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
36.29k stars 14.09k forks source link

Conflicting results between Ruff linting and Pydantic tests #41847

Open CYarros10 opened 2 weeks ago

CYarros10 commented 2 weeks ago
          Conflicting tests when evaluating `airflow/tests/providers/google/cloud/operators/vertex_ai/test_supervised_fine_tuning.py` and `airflow/tests/providers/google/cloud/hooks/vertex_ai/test_supervised_fine_tuning.py` ...

Static checks, mypy, docs / Static checks :

Run 'ruff' for extremely fast Python linting.......................................Failed
- hook id: ruff
- exit code: 1

All checks passed!
tests/providers/google/cloud/hooks/vertex_ai/test_supervised_fine_tuning.py:27:1: E402 Module level import not at top of file
   |
25 |   pytest.importorskip("google.cloud.aiplatform_v1")
26 |   
27 | / from airflow.providers.google.cloud.hooks.vertex_ai.supervised_fine_tuning import (
28 | |     SupervisedFineTuningHook,
29 | | )
   | |_^ E402
30 |   from tests.providers.google.cloud.utils.base_gcp_mock import (
31 |       mock_base_gcp_hook_default_project_id,

Special tests / Pydantic removed test / All:Pydantic-Removed-Postgres:12:3.8: API Always BranchExternalPython BranchPythonVenv CLI Core ExternalPython Operators Other PlainAsserts :

_ ERROR collecting tests/providers/google/cloud/hooks/vertex_ai/test_supervised_fine_tuning.py _
ImportError while importing test module '/opt/airflow/tests/providers/google/cloud/hooks/vertex_ai/test_supervised_fine_tuning.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/providers/google/cloud/hooks/vertex_ai/test_supervised_fine_tuning.py:24: in <module>
    from airflow.providers.google.cloud.hooks.vertex_ai.supervised_fine_tuning import (
airflow/providers/google/cloud/hooks/vertex_ai/supervised_fine_tuning.py:25: in <module>
    import vertexai
E   ModuleNotFoundError: No module named 'vertexai'

_Originally posted by @CYarros10 in https://github.com/apache/airflow/pull/41807#discussion_r1735221087_

CYarros10 commented 2 weeks ago

I'd imagine this is a general issue regarding how Ruff formatting/linting checks are required and how pytest.importorskip is sometimes necessary before other imports to pass Pydantic environment build tests - example:

# For no Pydantic environment, we need to skip the tests
pytest.importorskip("google.cloud.aiplatform_v1")

from airflow.providers.google.cloud.operators.vertex_ai.supervised_fine_tuning import (
    SupervisedFineTuningTrainOperator,
)