search

kedro-org / kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
https://kedro.org
Apache License 2.0
10.03k stars 906 forks source link

Add official support for Python 3.13 #4267

Closed astrojuanlu closed 6 days ago

astrojuanlu commented 4 weeks ago

Description

Just tested the Spaceflights starter on Python 3.13 and everything works smoothly.

See https://github.com/kedro-org/kedro/pull/3587 for the addition of official Python 3.12 support.

astrojuanlu commented 3 weeks ago

This will be the parent issue tracking all repos. The upgrade most likely needs to happen in a certain order.

noklam commented 2 weeks ago

https://github.com/kedro-org/kedro/actions/runs/11666306391/job/32480967172?pr=4276

Adding some info here, seems like it breaks some micropkg command in 3.13 Windows

ravi-kumar-pilla commented 1 week ago

Hi Team,

There are some blockers adding python 3.13 support for kedro-plugins

  1. Kedro-airflow - blocked by apache-airflow support for python 3.13 ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (google-re2, msgspec) Related issues as I could not find any active discussion in apache-airflow for python 3.13 - Google re2 : Open issues - https://github.com/google/re2/issues/516 Msgspec : https://github.com/jcrist/msgspec/issues/764

  2. Kedro-datasets - blocked by tensorflow

    Tensorflow - https://github.com/tensorflow/tensorflow/issues/78774

Thank you

noklam commented 1 week ago

Just to clarify, we can still do a release for kedro and kedro-datasets without python 3.13 support right? We break kedro-datasets exactly for this reason so I don't see that as a blocking factor.

ravi-kumar-pilla commented 1 week ago

Just to clarify, we can still do a release for kedro and kedro-datasets without python 3.13 support right? We break kedro-datasets exactly for this reason so I don't see that as a blocking factor.

I did not fully understand what you meant by a release without python 3.13 support. So python 3.13 support is merged in Kedro but for kedro-datasets the build is failing as Tensorflow does not yet support python 3.13 and our build depends on that.

ravi-kumar-pilla commented 1 week ago

Hi Team,

Blockers for python 3.13 support on -

Kedro-plugins:

Kedro-starters: The PR is merged but waiting for 3.13.1 patch release

Thank you

merelcht commented 6 days ago

Re kedro-starters: this is just for adding 3.13 builds to the starters, but doesn't make any changes in the code, so we're still good on releasing kedro with 3.13 support. It's also fine to wait with kedro-airflow and kedro-datasets. I suggest creating separate tickets for all of these so we can track them and the blockers individually.

ravi-kumar-pilla commented 6 days ago

Closing this in favor of - https://github.com/kedro-org/kedro-plugins/issues/940 and https://github.com/kedro-org/kedro-starters/issues/259