hofstadter-io / hof

Framework that joins data models, schemas, code generation, and a task engine. Language and technology agnostic.
https://docs.hofstadter.io
Apache License 2.0
511 stars 35 forks source link

hof/flow -> airflow #389

Open verdverm opened 3 months ago

verdverm commented 3 months ago

It would be great to transform workflows to other frameworks, airflow as an example, to scale out. There by, hof flow becomes a way to quickly develop the ETL, re: better CUE ergonomics for the ETL twiddling / iteration

Probably helpful if we could snapshot task i/o too, for both iteration and determining what code to generate. Ideally flow can have a standardize record format to feed into hof generators for target workflow frameworks. Might also be helpful for non-workflow functions, like API handlers when generating CRUD+