Support for other DataFrame libraries

cmudig / AutoProfiler

Automatically profile dataframes in the Jupyter sidebar

BSD 3-Clause "New" or "Revised" License

341 stars 11 forks source link

Support for other DataFrame libraries #147

Open lucas-nelson-uiuc opened 6 months ago

lucas-nelson-uiuc commented 6 months ago

Wondering if there's been discussion for supporting other dataframe (or pandas-compatible) libraries? Happy to talk through or help develop support to extend the ecosystem.

willeppy commented 6 months ago

Hey thanks for starting discussion on this, I think would be really cool. A couple of libraries that come to mind, curious which ones you or others are using the most that would be the best to support

Polars
Ibis
Modin -- this might work with minimal overhead since has exact same API as Pandas as my understanding
Raw numpy matricies (if 2d)

willeppy commented 6 months ago

Generally, in terms of implementation I think would involve...

Python pandas executor to support different libraries and detect different dataframe types in the front end code that is executed (https://github.com/cmudig/AutoProfiler/blob/main/src/dataAPI/jupyter/PythonExecutor.ts)
And then implementing the required functions in python like we have for pandas (https://github.com/cmudig/AutoProfiler/blob/main/digautoprofiler/profile_lib.py)