LamaAni / KubernetesJobOperator

An airflow operator that executes a task in a kubernetes cluster, given a kubernetes yaml configuration or an image refrence.
57 stars 8 forks source link

FEATURE: need 'templates_dict' to ensure proper rendering of time macros. #95

Open kuixiang opened 7 months ago

kuixiang commented 7 months ago

When using KubernetesJobOperator in Apache Airflow and encountering issues where Jinja templating for jinja_job_args is treating Airflow time macros as string, shall we provide a mechanism similar to the templates_dict in KubernetesPodOperator? e.g.

from datetime import datetime, timedelta
from airflow import DAG
from airflow.operators.kubernetes_pod_operator import KubernetesPodOperator

default_args = {
    'owner': 'airflow',
    'start_date': datetime(2024, 1, 1),
    'retries': 1,
    'retry_delay': timedelta(minutes=5),
}

dag = DAG(
    'my_dag',
    default_args=default_args,
    schedule_interval='@daily',
)

my_task = KubernetesPodOperator(
    task_id='my_task',
    namespace='my_namespace',
    image='my_image:latest',
    cmds=['/bin/bash', '-c'],
    arguments=['echo', '{{ ds }}'],  # Using time macro directly
    templates_dict={'my_jinja_template': '{{ ds }}'},  # Template for Jinja templating
    dag=dag,
)
LamaAni commented 7 months ago

Hi, Sorry I dont understand what you mean. Do you want to extend the jinja template dict? If so, feel free to send in a PR.

LamaAni commented 7 months ago

Any more info on this?