fugue-project / fugue

A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.
https://fugue-tutorials.readthedocs.io/
Apache License 2.0
2.01k stars 94 forks source link

[DEPRECATION] `as_ibis` in FugueWorkflow #505

Closed goodwanghan closed 10 months ago

goodwanghan commented 1 year ago

This is NOT to deprecate Ibis usage from Fugue

Currently, we have two ways to use Ibis, one way is to create the dataframe and execution engine abstractions on top of Ibis, and this is used by our Bigquery and trino integrations. Another way is inside FugueWorkflow we enable a function called as_ibis to convert a FugueWorkflowDataFrame in to Ibis, this is a transition between two different dags.

This feature is rarely used by users, plus, we want to hide FugueWorkflow from the top level APIs, so maintaining the second use case no longer has value. So, to make Fugue more lightweight and more focused on useful feature, we will remove this feature.