search

kedro-org / kedro-plugins

First-party plugins maintained by the Kedro team.
Apache License 2.0
92 stars 89 forks source link

Bump dask version to the latest #278

Closed jw-cpnet closed 1 year ago

jw-cpnet commented 1 year ago

The dask version is pinned to 2021.12.0 and pandas is currently on version 2.

This dask version doesn't work with pandas 2.

With any pandas dataframe, if you run following script:

ddf = dd.from_pandas(df, npartitions=1)

you will get an error:

AttributeError: module 'pandas.core.strings' has no attribute 'StringMethods'

According to dask's changelog, the latest dask supports pandas 2:

https://docs.dask.org/en/stable/changelog.html

Is there any plan to bump dask version to the latest?

noklam commented 1 year ago

I am open to bump dask, would you like to create a PR for that?

jw-cpnet commented 1 year ago

sure. https://github.com/kedro-org/kedro-plugins/pull/282