search

astronomer / dag-factory

Dynamically generate Apache Airflow DAGs from YAML configuration files
Apache License 2.0
1.21k stars 183 forks source link

[Test failure] ImportError: cannot import name 'soft_unicode' from 'markupsafe' - Airflow 2.2 #238

Closed pankajkoti closed 1 month ago

pankajkoti commented 1 month ago

Affecting test matrix job with Airflow 2.2

Observed in PR: #237

Failed GitHub action https://github.com/astronomer/dag-factory/actions/runs/11288449566/job/31396237227?pr=237

_____________________ ERROR collecting tests/test_utils.py _____________________
ImportError while importing test module '/home/runner/work/dag-factory/dag-factory/tests/test_utils.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../.local/share/hatch/env/virtual/dag-factory/onaeQMJR/tests.py3.8-2.2/lib/python3.8/site-packages/_pytest/python.py:493: in importtestmodule
    mod = import_path(
../../../.local/share/hatch/env/virtual/dag-factory/onaeQMJR/tests.py3.8-2.2/lib/python3.8/site-packages/_pytest/pathlib.py:582: in import_path
    importlib.import_module(module_name)
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1014: in _gcd_import
    ???
<frozen importlib._bootstrap>:991: in _find_and_load
    ???
<frozen importlib._bootstrap>:975: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:671: in _load_unlocked
    ???
../../../.local/share/hatch/env/virtual/dag-factory/onaeQMJR/tests.py3.8-2.2/lib/python3.8/site-packages/_pytest/assertion/rewrite.py:[184](https://github.com/astronomer/dag-factory/actions/runs/11288449566/job/31396237227?pr=237#step:6:185): in exec_module
    exec(co, module.__dict__)
tests/test_utils.py:7: in <module>
    from dagfactory import utils
dagfactory/__init__.py:3: in <module>
    from .dagfactory import DagFactory, load_yaml_dags
dagfactory/dagfactory.py:10: in <module>
    from airflow.configuration import conf as airflow_conf
../../../.local/share/hatch/env/virtual/dag-factory/onaeQMJR/tests.py3.8-2.2/lib/python3.8/site-packages/airflow/__init__.py:34: in <module>
    from airflow import settings
../../../.local/share/hatch/env/virtual/dag-factory/onaeQMJR/tests.py3.8-2.2/lib/python3.8/site-packages/airflow/settings.py:35: in <module>
    from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf  # NOQA F401
../../../.local/share/hatch/env/virtual/dag-factory/onaeQMJR/tests.py3.8-2.2/lib/python3.8/site-packages/airflow/configuration.py:37: in <module>
    from airflow.exceptions import AirflowConfigException
../../../.local/share/hatch/env/virtual/dag-factory/onaeQMJR/tests.py3.8-2.2/lib/python3.8/site-packages/airflow/exceptions.py:26: in <module>
    from airflow.api_connexion.exceptions import NotFound as ApiConnextionNotFound
../../../.local/share/hatch/env/virtual/dag-factory/onaeQMJR/tests.py3.8-2.2/lib/python3.8/site-packages/airflow/api_connexion/exceptions.py:21: in <module>
    from airflow._vendor.connexion import FlaskApi, ProblemException, problem
../../../.local/share/hatch/env/virtual/dag-factory/onaeQMJR/tests.py3.8-2.2/lib/python3.8/site-packages/airflow/_vendor/connexion/__init__.py:5: in <module>
    from .apis import AbstractAPI  # NOQA
../../../.local/share/hatch/env/virtual/dag-factory/onaeQMJR/tests.py3.8-2.2/lib/python3.8/site-packages/airflow/_vendor/connexion/apis/__init__.py:1: in <module>
    from .abstract import AbstractAPI  # NOQA
../../../.local/share/hatch/env/virtual/dag-factory/onaeQMJR/tests.py3.8-2.2/lib/python3.8/site-packages/airflow/_vendor/connexion/apis/abstract.py:16: in <module>
    from ..spec import Specification
../../../.local/share/hatch/env/virtual/dag-factory/onaeQMJR/tests.py3.8-2.2/lib/python3.8/site-packages/airflow/_vendor/connexion/spec.py:5: in <module>
    import jinja2
../../../.local/share/hatch/env/virtual/dag-factory/onaeQMJR/tests.py3.8-2.2/lib/python3.8/site-packages/jinja2/__init__.py:12: in <module>
    from .environment import Environment
../../../.local/share/hatch/env/virtual/dag-factory/onaeQMJR/tests.py3.8-2.2/lib/python3.8/site-packages/jinja2/environment.py:25: in <module>
    from .defaults import BLOCK_END_STRING
../../../.local/share/hatch/env/virtual/dag-factory/onaeQMJR/tests.py3.8-2.2/lib/python3.8/site-packages/jinja2/defaults.py:3: in <module>
    from .filters import FILTERS as DEFAULT_FILTERS  # noqa: F401
../../../.local/share/hatch/env/virtual/dag-factory/onaeQMJR/tests.py3.8-2.2/lib/python3.8/site-packages/jinja2/filters.py:13: in <module>
    from markupsafe import soft_unicode
E   ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/home/runner/.local/share/hatch/env/virtual/dag-factory/onaeQMJR/tests.py3.8-2.2/lib/python3.8/site-packages/markupsafe/__init__.py)
pankajkoti commented 1 month ago

Resolved in #237 by pre-installing the CNCF K8S provider using Airflow constraints.