search

indygreg / python-build-standalone

Produce redistributable builds of Python
Mozilla Public License 2.0
1.98k stars 127 forks source link

Add ppc64le support #165

Closed clnperez closed 1 year ago

clnperez commented 1 year ago

Adding support for ppc64le. As with s390x support (added 7a0c31d3f365fed4a8f0e3125ce870c9afb07618), this only needed additions for the build triples and a new build target defined. I'd like to get this added to use with the envoy builds. (See https://github.com/envoyproxy/envoy/issues/25090).

indygreg commented 1 year ago

I haven’t looked at the code yet, but missing platform additions are very much welcomed and appreciated! So let me know when this is ready to review and we’ll get it merged.

ninehills commented 1 year ago

I am also working on the porting of the envoy to the ppc64le architecture. Next, I will test this PR.~

I find some bug in this pr, and fix it in:

https://github.com/clnperez/python-build-standalone/pull/1

Then trigger the github action:

https://github.com/ninehills/python-build-standalone/actions/runs/4361856285

ninehills commented 1 year ago

Local build successfully. @clnperez you can merge my pr.

clnperez commented 1 year ago

thanks @ninehills!

clnperez commented 1 year ago

@indygreg happy to hear that! i merged @ninehills's changes into this PR. Would you rather me squash and re-push? Or will you just do a squash & merge?

ninehills commented 1 year ago

I found that some of the validation and publishing logic still needs to be modified. I will continue testing.

ninehills commented 1 year ago

github workflow run successfully: https://github.com/ninehills/python-build-standalone/actions/runs/4371696198

pr already pushed.

clnperez commented 1 year ago

removing from WIP status, as @ninehills has tested and the workflow ran successfully on my fork. PTAL @indygreg and lmk how you want me to handle the multiple commits. thanks!

clnperez commented 1 year ago

@indygreg can you take a look at this one?

ninehills commented 1 year ago

It seems that CPython can only accept powerpc64le-linux-gnu during compilation (possibly referencing the architecture naming of gcc), but the standard architecture name we built should be ppc64le according to the standard.

https://github.com/python/cpython/blob/5e6661bce968173fa45b74fa2111098645ff609c/configure.ac#L1011

Other discussions: https://github.com/pypa/manylinux/issues/687

clnperez commented 1 year ago

@ninehills how did you package & test this with envoy when you built it? i can cross-compile just fine, but am having no luck figuring out what was built and how it is turned into a consumable release archive.

ninehills commented 1 year ago

@ninehills how did you package & test this with envoy when you built it? i can cross-compile just fine, but am having no luck figuring out what was built and how it is turned into a consumable release archive.

https://github.com/ninehills/python-build-standalone/actions/runs/4524288535

Now build is success.

clnperez commented 1 year ago

@indygreg can you weigh in?

i can see from the github workflow that the archives are built -- but when I run just the power cross-build (./build-linux.py --target-triple ppc64le-unknown-linux-gnu) -- i don't get one. so i tried ./build-linux.py --target-triple ppc64le-unknown-linux-gnu --python cpython-3.11 --optimizations noopt and then I get: compressing Python archive to /home/christy/go/src/github.com/indygreg/python-build-standalone/dist/cpython-3.11.1-ppc64le-unknown-linux-gnu-noopt-20230309T0934.tar.zst

It doesn't seem that the type of build via build-flags would be the deciding factor on whether or not something shows up in /dist.

indygreg commented 1 year ago

It seems like ppc64le is what's in PEP 599 / 600 and was the historically accurate target name. But powerpc64le seems to be what people want to transition to. And CPython's configure.ac appears to insist on powerpc64le.

So I think it makes sense to keep go with powerpc64le since it will be future compatible. But I wouldn't be surprised if we run into issues installing binary wheels when using this platform name since ppc64le was historically used. 🤷‍♂️

clnperez commented 1 year ago

thanks @indygreg! it is a messy space for sure 🤞🏻

clnperez commented 1 year ago

oh -- do i need to squash these commits? anything else to merge?

indygreg commented 1 year ago

I have this cherry picked into my local repo. Will push this to main once CI passes.

Thanks for your hard work and patience!

clnperez commented 1 year ago

Thanks @indygreg. Does this project have a release schedule we should wait for, or can we just request a new one be made so we can add this to the envoy builds?

indygreg commented 1 year ago

There's no formal release schedule. I try to cut releases after new Python versions, important security vulns, or other major changes. We're several weeks overdue for a release. Please create a new issue to track the next release. Hopefully I'll find time in the next 1-2 weeks.

clnperez commented 1 year ago

adding to rules_python via https://github.com/bazelbuild/rules_python/pull/1234. thanks for the release @indygreg 🎉