Bump s3fs version number

kedro-org / kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.

https://kedro.org

Apache License 2.0

9.91k stars 900 forks source link

Bump s3fs version number #1995

Closed datajoely closed 10 months ago

datajoely commented 1 year ago

Description

Is your feature request related to a problem? A clear and concise description of what the problem is: "I'm always frustrated when ..."

Our current setup.py has the following restriction S3FS = "s3fs>=0.3.0, <0.5"

s3fs has been on calendar versioning since April 2021 so we're quite out of date.

Implementation

Move to the same calendar versioning scheme we apply to related fsspec project.
Fix tests using moto or aiobotocore

noklam commented 1 year ago

See #2017 for more details. This may not be trivial as bumping s3fs immediately cause a lot of tests fails. I suspect this is a pure test issue (Kedro will still work, but the tests are not working because of moto or aiobotocore). Will confirm this before backlog grooming.

Estimate: 5/8

noklam commented 1 year ago

@datajoely There is some hope that we can tackle it easily with newer version of moto without any change in code.

astrojuanlu commented 11 months ago

xref https://github.com/kedro-org/kedro-plugins/issues/337, which has an open PR https://github.com/kedro-org/kedro-plugins/pull/348

merelcht commented 10 months ago

this was completed in https://github.com/kedro-org/kedro-plugins/pull/463 and #3405