Consider leveraging Dask instead of Pandas where possible

unit8co / darts

A python library for user-friendly forecasting and anomaly detection on time series.

https://unit8co.github.io/darts/

Apache License 2.0

8.05k stars 878 forks source link

Consider leveraging Dask instead of Pandas where possible #1241

Open hrzn opened 2 years ago

hrzn commented 2 years ago

Xarray already integrate Dask (see: https://docs.xarray.dev/en/stable/user-guide/dask.html).

We could perhaps use Dask dataframes (instead of pandas) e.g., in RegressionModels, to speedup computation, and perhaps even at some point offer distributed computing to the user for RegressionModel (because Dask also integrates with sklearn).

martinb-ai commented 2 years ago

Dask would be very useful when needing to compute on MUCH larger datasets!