search

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
35.41k stars 13.83k forks source link

Rewrite our Github Action workflows to Python #13182

Closed potiuk closed 1 year ago

potiuk commented 3 years ago

Yesterday (18th of December) GitHub announced a 3rd-party Python GHApi: https://github.blog/2020-12-18-learn-about-ghapi-a-new-third-party-python-client-for-the-github-api/

One of the features of the API is the ability to write GitHub Actions Workflows (including custom actions) entirely in Python:

https://github.blog/2020-12-18-learn-about-ghapi-a-new-third-party-python-client-for-the-github-api/#github-actions

This is super cool. Knowing the difficulties of combining YAML files, bash scripts and TypeScripts Actions) together, this is a so much better and easier way of writing the workflows,

We should rewrite all our workflows using this new API !

potiuk commented 3 years ago

cc: @kaxil @ashb @turbaszek @mik-laj and @TobKed - let's do it!

kaxil commented 3 years ago

🎉 Yes let's do it

emmanue commented 2 years ago

@potiuk I will like to help and contribute.

TobKed commented 2 years ago

@potiuk is it sill something considered to be done?

eladkal commented 2 years ago

@potiuk is it sill something considered to be done?

This is in fact in progress. We have an intern from the Outreachy project that will take over this.

potiuk commented 2 years ago

Oh yeah. I will update the issue soon with an inventory of what is needed and we will work from here. Similarly https://github.com/apache/airflow/issues/12282 with rewriting breeze. Those will have some overlap so the two interns will be working together. More on that soon!

potiuk commented 2 years ago

Yeah. Finally the time had come to start work on the rewrite:

First, founding PR is here: https://github.com/apache/airflow/pull/19867/ - those who are interested will find there some ADR (Architecture Decision Records) which we will keep throughout the whole project to capture all important decisions we've made and once we merge it, more stuff will come.

eladkal commented 1 year ago

I believe this is now completed :) ?

potiuk commented 1 year ago

Oh. Vrery much so :).