kedro-org / kedro-plugins

First-party plugins maintained by the Kedro team.
Apache License 2.0
92 stars 89 forks source link

Wrong version of pinned dependency Kedro in Kedro-Datasets #436

Closed javihernaxpo closed 10 months ago

javihernaxpo commented 11 months ago

Description

The version of Kedro in dependencies is not correct to use Kedro-Datasets.

Context

The version specified is Kedro>=0.16. Testing with kedro version 0.17.7 and python 3.8, I get the following error when trying to import the pandas.ParquetDataset:

from kedro_datasets.pandas.parquet_dataset import ParquetDataset
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jovyan/my-conda-envs/fact/lib/python3.8/site-packages/kedro_datasets/pandas/parquet_dataset.py", line 26, in <module>
    class ParquetDataset(AbstractVersionedDataset[pd.DataFrame, pd.DataFrame]):
TypeError: 'ABCMeta' object is not subscriptable

It should be pinned to Kedro >=0.18.2

Steps to Reproduce

  1. python version 3.8
  2. kedro version 0.17.7
  3. from kedro_datasets.pandas.parquet_dataset import ParquetDataset

Expected Result

It corrects imports the class

Actual Result

Fails with error: TypeError: 'ABCMeta' object is not subscriptable

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jovyan/my-conda-envs/fact/lib/python3.8/site-packages/kedro_datasets/pandas/parquet_dataset.py", line 26, in <module>
    class ParquetDataset(AbstractVersionedDataset[pd.DataFrame, pd.DataFrame]):
TypeError: 'ABCMeta' object is not subscriptable```

-- Separate them if you have more than one.



## Your Environment
Include as many relevant details about the environment in which you experienced the bug:

* Kedro version used (`pip show kedro` or `kedro -V`): 0.17.7
* Kedro plugin and kedro plugin version used (`pip show kedro-airflow`): kedro-datasets=1.8.0
* Python version used (`python -V`): Python 3.8.17 
* Operating system and version: Ubuntu 20.04.4 LTS
astrojuanlu commented 11 months ago

Thanks @javihernaxpo for reporting, you're right:

https://github.com/kedro-org/kedro-plugins/blob/f59e93013367c7a3808db3f9296e3cab1378a5c3/kedro-datasets/pyproject.toml#L13-L14

We should fix it. Would you like to send a pull request for it?

astrojuanlu commented 11 months ago

After giving this a second look it's not so obvious what's going on here, we deliberately relaxed the pinning because it was problematic and we checked compatibility back then. We'll have to dig a bit more.

merelcht commented 10 months ago

Thanks for flagging this @javihernaxpo, however we deliberately made the pinning less strict as it was causing lots of issues for users with older Kedro versions. See this conversation from https://github.com/kedro-org/kedro/issues/2409#issuecomment-1466992742. We will not change this approach to allow our users flexibility with their Kedro versions.