DAGs are not imported in webserver, only in scheduler

[clementchane]

Checks

• [X] I have checked for existing issues.
• [X] This report is about the User-Community Airflow Helm Chart.

Chart Version

airflow-1.8.0, App version 2.5.1

Kubernetes Version

Client Version: v1.26.0
Kustomize Version: v4.5.7
Server Version: v1.23.8

Helm Version

version.BuildInfo{Version:"v3.10.1", GitCommit:"9f88ccb6aee40b9a0535fcc7efea6055e1ef72c9", GitTreeState:"clean", GoVersion:"go1.18.7"}

Description

Hello,

I am new to Airflow so I might be doing something wrong.

In the Airflow UI, I found out that only DAGs in the scheduler pod are synced. When I manually add DAG files in the webserver pod, they do not appear in the UI.

My use case is that I want to upload dags from the airflow UI and I found this plugin: https://github.com/andreax79/airflow-code-editor, which only upload files to the webserver pod.

Is that a bug, is there something I should configure in the scheduler?

Thank you for your help.

Here is my example dag :

from airflow import DAG
from airflow.operators.python_operator import PythonOperator
from datetime import datetime, timedelta

default_args = {
    'owner': 'me',
    'start_date': datetime(2020, 1, 1),
    'depends_on_past': False,
    'retries': 1,
    'retry_delay': timedelta(minutes=5),
}
dag = DAG(
    'my_dag_id',
    default_args=default_args,
    schedule_interval=timedelta(minutes=10),
)
def my_function():
    print("Hello World!")
task = PythonOperator(
    task_id='my_task_id',
    python_callable=my_function,
    dag=dag,
)

Relevant Logs

No response

Custom Helm Values

No response

[thesuperzapper]

@clementchane I just want to confirm you resolved your issue?

[clementchane]

Hello, Yes sorry I didnt know I had to run the scheduler: I ran on the airflow webserver : "nohup airflow scheduler &" and it works Is there a better way to do that with config maybe?

Actually I m still working on it because I have another issue. I have a failing DAG but I dont have any logs:

Do you happen to know why? (I havent configured anything for logs, so logs should be default inside the pod) I know I should probably create a new issue, but I m still looking into the problem on my side.

Thank you for your help ! Clement

[thesuperzapper]

@clementchane Firstly, are you sure you are using the chart from this repo (the "User Community" one), because I think you might be using the one from the apache/airflow repo.

But either way, both charts will run the scheduler for you (in its own Pod), so something else is going wrong.