aws / amazon-mwaa-docker-images

Apache License 2.0
24 stars 11 forks source link

Enabling Sentry in Airflow 2.9.2 Causes Failures #121

Closed rafidka closed 1 month ago

rafidka commented 2 months ago

Describe the bug

Enabling Sentry in Airflow 2.9.2 causes failures.


To Reproduce Steps to reproduce the behavior:

  1. In requirements.txt, specify single dependency: apache-airflow[sentry]==2.9.2
  2. Add the following Airflow configurations (either in a MWAA environment or using the Docker Compose of this repo): a. AIRFLOW__SENTRY__SENTRY_DSN = https://foo@sentry.io/123 b. AIRFLOW__SENTRY__SENTRY_ON = True

Start the environment. You will see errors like the following:

mwaa-292-scheduler  | [2024-07-21T18:16:39.026+0000] {cmd.py:85} INFO - Traceback (most recent call last):
mwaa-292-scheduler  | [2024-07-21T18:16:39.027+0000] {cmd.py:85} INFO - File "/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/logging_config.py", line 41, in configure_logging
mwaa-292-scheduler  | [2024-07-21T18:16:39.027+0000] {cmd.py:85} INFO - logging_config = import_string(logging_class_path)
mwaa-292-scheduler  | [2024-07-21T18:16:39.027+0000] {cmd.py:85} INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
mwaa-292-scheduler  | [2024-07-21T18:16:39.027+0000] {cmd.py:85} INFO - File "/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/utils/module_loading.py", line 39, in import_string
mwaa-292-scheduler  | [2024-07-21T18:16:39.027+0000] {cmd.py:85} INFO - module = import_module(module_path)
mwaa-292-scheduler  | [2024-07-21T18:16:39.027+0000] {cmd.py:85} INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^
mwaa-292-scheduler  | [2024-07-21T18:16:39.028+0000] {cmd.py:85} INFO - File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
mwaa-292-scheduler  | [2024-07-21T18:16:39.028+0000] {cmd.py:85} INFO - return _bootstrap._gcd_import(name[level:], package, level)
mwaa-292-scheduler  | [2024-07-21T18:16:39.028+0000] {cmd.py:85} INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
mwaa-292-scheduler  | [2024-07-21T18:16:39.028+0000] {cmd.py:85} INFO - File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
mwaa-292-scheduler  | [2024-07-21T18:16:39.028+0000] {cmd.py:85} INFO - File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
mwaa-292-scheduler  | [2024-07-21T18:16:39.028+0000] {cmd.py:85} INFO - File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
mwaa-292-scheduler  | [2024-07-21T18:16:39.028+0000] {cmd.py:85} INFO - File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
mwaa-292-scheduler  | [2024-07-21T18:16:39.028+0000] {cmd.py:85} INFO - File "<frozen importlib._bootstrap_external>", line 940, in exec_module
mwaa-292-scheduler  | [2024-07-21T18:16:39.029+0000] {cmd.py:85} INFO - File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
mwaa-292-scheduler  | [2024-07-21T18:16:39.029+0000] {cmd.py:85} INFO - File "/python/mwaa/logging/config.py", line 33, in <module>
mwaa-292-scheduler  | [2024-07-21T18:16:39.029+0000] {cmd.py:85} INFO - from mwaa.logging import cloudwatch_handlers
mwaa-292-scheduler  | [2024-07-21T18:16:39.029+0000] {cmd.py:85} INFO - File "/python/mwaa/logging/cloudwatch_handlers.py", line 18, in <module>
mwaa-292-scheduler  | [2024-07-21T18:16:39.029+0000] {cmd.py:85} INFO - from airflow.models.taskinstance import TaskInstance
mwaa-292-scheduler  | [2024-07-21T18:16:39.029+0000] {cmd.py:85} INFO - File "/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/models/taskinstance.py", line 100, in <module>
mwaa-292-scheduler  | [2024-07-21T18:16:39.029+0000] {cmd.py:85} INFO - from airflow.sentry import Sentry
mwaa-292-scheduler  | [2024-07-21T18:16:39.030+0000] {cmd.py:85} INFO - File "/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/sentry.py", line 58, in <module>
mwaa-292-scheduler  | [2024-07-21T18:16:39.030+0000] {cmd.py:85} INFO - import sentry_sdk
mwaa-292-scheduler  | [2024-07-21T18:16:39.030+0000] {cmd.py:85} INFO - ModuleNotFoundError: No module named 'sentry_sdk'
mwaa-292-scheduler  | [2024-07-21T18:16:39.030+0000] {cmd.py:85} INFO - 
mwaa-292-scheduler  | [2024-07-21T18:16:39.030+0000] {cmd.py:85} INFO - During handling of the above exception, another exception occurred:
mwaa-292-scheduler  | [2024-07-21T18:16:39.030+0000] {cmd.py:85} INFO - 
mwaa-292-scheduler  | [2024-07-21T18:16:39.030+0000] {cmd.py:85} INFO - Traceback (most recent call last):
mwaa-292-scheduler  | [2024-07-21T18:16:39.030+0000] {cmd.py:85} INFO - File "/usr/local/airflow/.local/bin/airflow", line 5, in <module>
mwaa-292-scheduler  | [2024-07-21T18:16:39.031+0000] {cmd.py:85} INFO - from airflow.__main__ import main
mwaa-292-scheduler  | [2024-07-21T18:16:39.031+0000] {cmd.py:85} INFO - File "/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/__init__.py", line 74, in <module>
mwaa-292-scheduler  | [2024-07-21T18:16:39.031+0000] {cmd.py:85} INFO - settings.initialize()
mwaa-292-scheduler  | [2024-07-21T18:16:39.031+0000] {cmd.py:85} INFO - File "/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/settings.py", line 531, in initialize
mwaa-292-scheduler  | [2024-07-21T18:16:39.031+0000] {cmd.py:85} INFO - LOGGING_CLASS_PATH = configure_logging()
mwaa-292-scheduler  | [2024-07-21T18:16:39.031+0000] {cmd.py:85} INFO - ^^^^^^^^^^^^^^^^^^^
mwaa-292-scheduler  | [2024-07-21T18:16:39.031+0000] {cmd.py:85} INFO - File "/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/logging_config.py", line 50, in configure_logging
mwaa-292-scheduler  | [2024-07-21T18:16:39.031+0000] {cmd.py:85} INFO - raise ImportError(f"Unable to load custom logging from {logging_class_path} due to {err}")
mwaa-292-scheduler  | [2024-07-21T18:16:39.032+0000] {cmd.py:85} INFO - ImportError: Unable to load custom logging from mwaa.logging.config.LOGGING_CONFIG due to No module named 'sentry_sdk'
mwaa-292-scheduler  | [2024-07-21T18:16:39.133+0000] {dblock.py:39} INFO - Releasing lock for airflow_db_init...
mwaa-292-scheduler  | [2024-07-21T18:16:39.134+0000] {dblock.py:42} INFO - Released lock for airflow_db_init
mwaa-292-scheduler  | Traceback (most recent call last):
mwaa-292-scheduler  |   File "<frozen runpy>", line 198, in _run_module_as_main
mwaa-292-scheduler  |   File "<frozen runpy>", line 88, in _run_code
mwaa-292-scheduler  |   File "/python/mwaa/entrypoint.py", line 725, in <module>
mwaa-292-scheduler  |     asyncio.run(main())
mwaa-292-scheduler  |   File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
mwaa-292-scheduler  |     return runner.run(main)
mwaa-292-scheduler  |            ^^^^^^^^^^^^^^^^
mwaa-292-scheduler  |   File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
mwaa-292-scheduler  |     return self._loop.run_until_complete(task)
mwaa-292-scheduler  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
mwaa-292-scheduler  |   File "/usr/local/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
mwaa-292-scheduler  |     return future.result()
mwaa-292-scheduler  |            ^^^^^^^^^^^^^^^
mwaa-292-scheduler  |   File "/python/mwaa/entrypoint.py", line 686, in main
mwaa-292-scheduler  |     await airflow_db_init(environ)
mwaa-292-scheduler  |   File "/python/mwaa/utils/dblock.py", line 102, in async_wrapper
mwaa-292-scheduler  |     return await func(*args, **kwargs)
mwaa-292-scheduler  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
mwaa-292-scheduler  |   File "/python/mwaa/entrypoint.py", line 114, in airflow_db_init
mwaa-292-scheduler  |     await run_command("airflow db migrate", env=environ)
mwaa-292-scheduler  |   File "/python/mwaa/utils/cmd.py", line 98, in run_command
mwaa-292-scheduler  |     raise CommandError(return_code, command)
mwaa-292-scheduler  | mwaa.utils.cmd.CommandError: Command 'airflow db migrate' exited with non-zero status 1

System Information


Expected behavior

The environment should not fail to start.


Screenshots

N/A


Additional context

N/A