search

samtecspg / mason

Data Operator Framework
Apache License 2.0
2 stars 3 forks source link

Airflow Scheduler Support #21

Open kyprifog opened 4 years ago

kyprifog commented 4 years ago

As the first async scheduler. Involves building mason airflow operator

kyprifog commented 4 years ago

The place to house this code (on the mason side) is already in master:

https://github.com/samtecspg/mason/blob/master/mason/clients/airflow/airflow_client.py#L16-L29

Though not operational as airflow isn't specified as valid scheduler anywhere: https://github.com/samtecspg/mason/blob/master/mason/examples/workflows/table/validated_infer/workflow.yaml#L34

kyprifog commented 4 years ago

As per our slack discussions about how the operator should function on the backend:

image