Closed LipuFei closed 1 month ago
I'm getting a related error message on Airflow 2.10 and Python 3.11
AttributeError: 'LocalKubernetesExecutor' object has no attribute 'slots_occupied'
CC: @o-nikolas -> seems to be result of #40017
I'll look into fixing this issue But I encourage folks to use Multiple Executor Configuration instead of these old (and should be deprecated) hybrid executors. These are exactly the types of issues that arise from these statically combined executors, since they're full of bespoke logic and hard coupling.
@potiuk what steps need to be followed for us to deprecate these old hybrid executors for airflow 3.0? Or is that not possible because they are in provider packages?
@potiuk what steps need to be followed for us to deprecate these old hybrid executors for airflow 3.0? Or is that not possible because they are in provider packages
We can deprecate-with-warnings those and remove them from providers when the time comes (which might or might not be faster than getting rid of Airflow 2 support from those providers.
On a related note ; the current helm chart (1.15.0) does not yet support the hybrid executor approach introduced in airflow 2.10.0.
In the helm chart there is a constraint on an ENUM of the "old" executors. Not sure what the best approach would be to validate the executor input (if possible) considering the possibility to specify your own custom executor.
@potiuk issue still exists on
Did you try with latest provider?
cc: @o-nikolas
@potiuk issue still exists on
- Python 3.11
- Airflow 2.10.1
please specify version of Celery provider that you are using
Apache Airflow version
Other Airflow 2 version (please specify below)
If "Other Airflow 2 version" selected, which one?
2.10.0 Python 3.12
What happened?
UPDATE: I think the
CeleryKubernetesExecutor
class should inheritBaseExecutor
instead ofLoggingMixin
.BaseExecutor
has theslots_occupied
method implemented.I get an error in the scheduler when I use
CeleryKubernetesExecutor
. My Airflow installation is on Kubernetes with the official Helm chart.What you think should happen instead?
No response
How to reproduce
Install Airflow with the official Helm chart with
CeleryKubernetesExecutor
and the scheduler container will raise this exception.Operating System
Kubernetes, Linux
Versions of Apache Airflow Providers
No response
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
Code of Conduct