jscheffl
commented
7 months ago We are running 2.8.1 and all is fine in this area on our side. From your descriptions I assume you can re-produce the error also with a fresh/clean setup? Have you tried one of the example DAGs delivered by Airflow? Or is the problem specific to your DAGs that you use (from previous version)? If it is on your DAGs maybe you need to post them. Otherwise, can you please post a bit more of the logs of the error? That is nothing I have seen recently.
luederm
potiuk
Apache Airflow version
2.8.1
If "Other Airflow 2 version" selected, which one?
No response
What happened?
After upgrading from airflow version 2.6 to 2.8, all DAGs I trigger through the UI fail immediately with the error:
ERROR - Executor reports task instance <TaskInstance: ... [queued]> finished (failed) although the task says it's queued. (Info: None) Was the task killed externally?I tested this with python version 3.9 and 3.11, and the issue persisted with both versions.
DAGs can be successfully ran by running them directly with
.test()What you think should happen instead?
No response
How to reproduce
On MacOS, create a conda environment with python 3.11:
conda create -n airflow-2-8-1 python=3.11Activate the environment and install google-re through conda:
conda activate airflow-2-8-1conda install google-re2Install airflow:
pip install "apache-airflow[async,celery,crypto,jdbc,ldap,password,mysql,postgres,redis,s3,sftp,ssh,slack]==2.8.1" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.8.1/constraints-3.11.txt"Set up airflow config and database.
Run airflow:
airflow standaloneTrigger an example DAG.
Operating System
MacOS Ventura 13.6
Versions of Apache Airflow Providers
No response
Deployment
Other
Deployment details
airflow was installed in a stand-alone conda environment. The command used to install airflow (when testing with python 3.11) is:
pip install "apache-airflow[async,celery,crypto,jdbc,ldap,password,mysql,postgres,redis,s3,sftp,ssh,slack]==2.8.1" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.8.1/constraints-3.11.txt"The package google-re2 was installed through conda prior to running this command because it fails to install through pip.
I created a new airflow database for testing the deployment. I tried running the airflow server with
airflow standaloneand by running each component separately.I have Airflow configured to use the LocalExecutor. I can successfully trigger tasks with the SequentialExecutor.
Anything else?
Output from conda list:
Are you willing to submit PR?
Code of Conduct