search

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
36.28k stars 14.08k forks source link

JSON Linter in Variables #27157

Open selvakumar-arumugapandian opened 1 year ago

selvakumar-arumugapandian commented 1 year ago

Description

JSON Linter in Variables

When user adds an object (JSON format) as variable, there should be option enabled to validate the object or validate automatically.

For example:

I have a variable named "my_var_group", with the following value

{
"dag_prefix": "my-prefix",
"dag_retry": 5
}

Imagine, I am using this variable "my_var_group" in 500+ DAGs and any mistake in the JSON fails to load all the 500+ DAGs.

So, A feature which does automatic linting of JSON object (By throwing error during variable creation/update) shall ensure that "error" prone JSON is not added.

Use case/motivation

Real-time experience in an environment where we are running more than 1000+ DAGs.

Related issues

No response

Are you willing to submit a PR?

Code of Conduct

boring-cyborg[bot] commented 1 year ago

Thanks for opening your first issue here! Be sure to follow the issue template!

bbovenzi commented 1 year ago

Good idea. Feel free to copy over the JSON linter we use for a dag run config: https://github.com/apache/airflow/blob/main/airflow/www/templates/airflow/trigger.html

selvakumar-arumugapandian commented 1 year ago

@bbovenzi , Thank you. I shall check this out.