We should create a feature for deployment admins to either allow or restrict users from using certain types of operators. For example, some deployment admins may want to restrict users from using KPOs, or only allow users to use custom operators that were built internally. This should be a configuration that is set at the deployment, not via dag code.
Use case/motivation
To avoid issues when upgrading and ensure that their users are following best practice, some of our customers want to restrict their users from using certain kinds of operators or only allow them to use specific ones, like for example using the databricks operator instead of the python operator. It's possible to write tests via pytest to check for this as part of cicd, but unless cicd is enforced and tests are imported from elsewhere during the cicd process, it is possible for users to get around this. We should have a first class way for people to add an allow/deny list via a proper configuration outside of dag code. Something like DAG_FACTORY__BANNED_OPERATORS='PythonOperator,KubernetesPodOperator'
Description
We should create a feature for deployment admins to either allow or restrict users from using certain types of operators. For example, some deployment admins may want to restrict users from using KPOs, or only allow users to use custom operators that were built internally. This should be a configuration that is set at the deployment, not via dag code.
Use case/motivation
To avoid issues when upgrading and ensure that their users are following best practice, some of our customers want to restrict their users from using certain kinds of operators or only allow them to use specific ones, like for example using the databricks operator instead of the python operator. It's possible to write tests via pytest to check for this as part of cicd, but unless cicd is enforced and tests are imported from elsewhere during the cicd process, it is possible for users to get around this. We should have a first class way for people to add an allow/deny list via a proper configuration outside of dag code. Something like
DAG_FACTORY__BANNED_OPERATORS='PythonOperator,KubernetesPodOperator'Related issues
No response
Are you willing to submit a PR?