Dynamic Versioning for Pandas

[NightHawk451]

🚀 Feature Request

Enable Dynamic version ranges of pandas, pyarrow and numpy.

🔈 Motivation

Dynamic versioning ensures that users can leverage the latest features, bug fixes and optimizations introduced in newer Pandas, Pyarrow, and Numpy releases. This not only enhances the overall performance and stability of the package, but allows users to seamlessly integrate it with their existing projects without being restricted to a specific Pandas version.

By pinning the pandas, pyarrow, and numpy version, we limit the capabilities of applications, specifically Machine Learning applications, that want to use this package.

pandas
pyarrow
numpy

Currently, Pandas is pinned to pandas==1.5.3 which prevents pandas 2.0. Pyarrow is pinned to 14.0.2 which prevents the current version of pyarrow (version 15) Numpy is less out of sync as the installed version is 1.24.4 while the current version is 1.26.0. However, this too will drift over time.

Unpinning these packages will alleviate these discrepancies. If it's too much to ask to unpin it, could you at least set a lower bound.

e.g.

pandas>=1.5.3
pyarrow>=14.0.2
numpy>=1.24.4

Thank you!

[github-actions[bot]]

Hello @NightHawk451, thank you for your interest in our work!

If this is a bug report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

[TheBeardedBerserkr]

+1

[ddl-gabrielhaim]

Hey @NightHawk451 I understand pinning dependencies is not great when you want to have flexibility to use latest packages.

We currently fix the versions because of dependencies in Domino Standard Environments which contains this and other Domino related package. Note that we are looking to upgrade our dependencies in upcoming versions.

Which version of Domino are you using? I could potentially look into creating a specific branch with relaxed requirements if you are ok installing from Github.

[LamDang]

+1 We are using Domino 5.8 and we use self managed Environment and these pinned packages are really limiting. What is the issue with allowing pandas >= 1.5.2?