[Python][Packaging] Support Python 3.13 and upload wheels

[jorisvandenbossche]

Python 3.13 is currently in a prerelease status (a first release candidate was released today: https://discuss.python.org/t/python-3-13-0-release-candidate-1-released/59703) and is expected to have a final release in October this year (https://devguide.python.org/versions).

We should ensure we are fully ready to support Python 3.13 for the PyArrow 18 release that will happen around the same time (or potentially even sooner if we would have a 17.0.x bug fix release).

https://github.com/apache/arrow/pull/42034 already fixed a build issue with Python 3.13, and on the short term I think we should try to have 1) a CI build using Python 3.13 to ensure this keeps working, and 2) set up a wheel build so we can upload nightly wheels for Python 3.13 for downstream testing.

(equivalent PR for Python 3.12: https://github.com/apache/arrow/pull/37901)

---

• [x] Start building nightly wheels for Python 3.13: https://github.com/apache/arrow/pull/43539
• [ ] Also add py 3.13 conda build once that is supported in conda-forge (https://github.com/apache/arrow/blob/712cfe6d84bd344cfe57a1e4c791f8a4d052c76d/dev/tasks/tasks.yml#L1200-L1201)
• [ ] Update requirements-wheel-build.txt and requirements-wheel-test.txt to install released versions
• [ ] Update the places where we currently use "3.13-rc" or "rc1" or "3.13.0-rc1" (dev/tasks/python-wheels/github.linux.yml, ci/scripts/install_python.sh, the windows docker files)
• [ ] Enable GCS testing once google-testbench (grpcio) can be installed (update install_gcs_testbench.sh) (or address GH-38183 and install testbench with independent python version)

[raulcd]

I was planning, if necessary, to delay the feature freeze for 18.0.0 for a week to have Python 3.13 wheels.

[jorisvandenbossche]

We can actually make wheels right now (and we should ideally produce nightly wheels well before the scheduled release of 18.0), because Python 3.13 is ABI stable now, and wheels built with the current Python 3.13 will work with the final 3.13 releases.

(so there is certainly no need to delay 18.0.0 AFAIK)

[jorisvandenbossche]

Follow-ups after https://github.com/apache/arrow/pull/43539:

• Also add py 3.13 conda build once that is supported in conda-forge (https://github.com/apache/arrow/blob/712cfe6d84bd344cfe57a1e4c791f8a4d052c76d/dev/tasks/tasks.yml#L1200-L1201)
• Update requirements-wheel-build.txt and requirements-wheel-test.txt to install released versions
• Update the places where we currently use "3.13-rc" or "rc1" or "3.13.0-rc1" (dev/tasks/python-wheels/github.linux.yml, ci/scripts/install_python.sh, the windows docker files)
• Enable GCS testing once google-testbench (grpcio) can be installed (update install_gcs_testbench.sh)

EDIT: added those to the top post