search

Kanaries / pygwalker

PyGWalker: Turn your pandas dataframe into an interactive UI for visual analysis
https://kanaries.net/pygwalker
Apache License 2.0
10.83k stars 549 forks source link

Would it be possbile to walk it with Datatable.frame type in the future? #510

Open plusmid opened 4 months ago

plusmid commented 4 months ago

https://github.com/h2oai/datatable It has higher performance reading and manipulating big csv data than pandas/polars/modin. But I can't walk this datatable.frame type.

ObservedObserver commented 4 months ago

Hi @plusmid, currently, pygwalker does not utilize the pandas API for dataframe-related computations.

When you pass a dataframe to pygwalker, it converts it into either a duckdb format or an in-browser memory JavaScript array. Therefore, the performance remains consistent across various dataframes.

We are exploring more native computation methods using the dataframe API. However, it appears that in most scenarios, using duckDB or integrating pygwalker with Snowflake yields quite good performance.