stackabletech / airflow-operator

Stackable Operator for Apache Airflow
Other
21 stars 2 forks source link

Logs from the airflow container are logged twice #473

Closed siegfriedweber closed 1 month ago

siegfriedweber commented 1 month ago

Logs from the airflow container are logged twice.

They are correctly written with the logging framework to the file airflow.py.json:

{
  "cluster": "airflow",
  "container": "airflow",
  "file": "airflow.py.json",
  "level": "INFO",
  "logger": "flask_appbuilder.baseviews",
  "message": "Registering route / ('GET',)",
  "namespace": "kuttl-test-heroic-badger",
  "pod": "airflow-scheduler-default-0",
  "role": "scheduler",
  "roleGroup": "default",
  "timestamp": "2024-07-15T17:53:18.924Z"
}

Additionally, the standard output is incorrectly captured (including the ANSI escape codes) and written to the file container.stdout.log:

{
  "cluster": "airflow",
  "container": "airflow",
  "file": "container.stdout.log",
  "level": "INFO",
  "logger": "ROOT",
  "message": "[\u001b[34m2024-07-15T17:53:18.924+0000\u001b[0m] {\u001b[34mbaseviews.py:\u001b[0m319} INFO\u001b[0m - Registering route / ('GET',)\u001b[0m",
  "namespace": "kuttl-test-heroic-badger",
  "pod": "airflow-scheduler-default-0",
  "role": "scheduler",
  "roleGroup": "default",
  "timestamp": "2024-07-15T17:53:19.882505453Z"
}

This problem was introduced in #311 in airflow_controller.rs#L748-L752.

The shell output must not be captured anymore but it must also be ensured that the logs of the Kubernetes executors are still gathered.