ibis-project / ibis

the portable Python dataframe library
https://ibis-project.org
Apache License 2.0
5.28k stars 595 forks source link

docs: guides on using Ibis with orchestration frameworks #7746

Closed lostmygithubaccount closed 9 months ago

lostmygithubaccount commented 11 months ago

Please describe the issue

we should document how to use Ibis with popular data orchestration frameworks, including but not limited to:

most of these should work out-of-the-box, but could use a simple how-to guide (e.g. setting up a TableIOManager in Dagster). for others, there may be useful frameworks (e.g. dbt-ibis or kedro-ibis-dataset)

I've never used Airflow, can probably do most of these in one go and create follow-up issues

I'm thinking of this as how-to guides, and perhaps an end-to-end getting started guide if people from those projects would be open to maintaining them

Code of Conduct

lostmygithubaccount commented 9 months ago

between https://ibis-project.org/posts/kedro-ibis/ (Kedro) and https://ibis-project.org/posts/ibis-analytics/ (Dagster) and https://ibis-project.org/posts/dbt-ibis/ (dbt) I'm going to call this done and close. I'm not sure how-to guides make sense for these on the Ibis documentation, we should probably try to get Ibis documented on the respective framework's documentation