fugue-project / fugue

A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.
https://fugue-tutorials.readthedocs.io/
Apache License 2.0
1.92k stars 94 forks source link

[DEPRECATION] `as_ibis` in FugueWorkflow #505

Closed goodwanghan closed 6 months ago

goodwanghan commented 10 months ago

This is NOT to deprecate Ibis usage from Fugue

Currently, we have two ways to use Ibis, one way is to create the dataframe and execution engine abstractions on top of Ibis, and this is used by our Bigquery and trino integrations. Another way is inside FugueWorkflow we enable a function called as_ibis to convert a FugueWorkflowDataFrame in to Ibis, this is a transition between two different dags.

This feature is rarely used by users, plus, we want to hide FugueWorkflow from the top level APIs, so maintaining the second use case no longer has value. So, to make Fugue more lightweight and more focused on useful feature, we will remove this feature.