vstinner
commented
4 years ago Having an ARM64 (aarch64) buildbot worker would be very useful. Python has a very large test suite. It's common to get tests failing only on one specific platform, compiler or architecture, or even worse: only one specific worker, unique (platform, compiler, architecture, Linux distribution) combo.
For example, right now I'm working on adding math.fma() function: it would be nice to be able to test if fma() behaves as "expected" on ARM64: https://github.com/python/cpython/pull/17987 ARM64 arch has VMLA, VMLS and VFMA instructions: not sure if libc fma() function uses it. That's the kind of thing that I would like to test functionally :-)
Note: Python has two 32-bit ARM buildbot workers:
- ARMv7 Debian buster: "Debian buster/sid armhf - Quad 2.3Ghz Cortex A15 2GiB (Nvidia Jetson Tegra K1) armv7l"
- (experimental) ARM32 Windows10: "Microsoft Windows 10 build machine with arm32 test runner"
vielmetti
stratakis
kind attention to @vstinner, @pablogsal @encukou @ncoghlan
Inspired by #187
Name, email, company, job title
Project Title and description
Which members of the community would benefit from your work?
Any consumer of the Python Programming Language targeting Linux on AArch64 and associated architectures.
While for the time being Python works and compiles mostly fine on Arm, there have been cases in the past where issues were discovered downstream and would have to be worked in-house and tested before submitting them upstream. And even then, there wasn't an easy way to test a fix.
The current status quo is that Arm is not an officially supported architecture by Python and I intend to work on changing that.
Having dedicated hardware will provide early testing for the Arm architecture while fostering a better relationship with the Python language and the Arm ecosystem.
The plan is to have VMs running on top of bare metal, each on a different flavor of Fedora and Red Hat Enterprise Linux/CentOS. The Fedora machines will provide feedback for the latest changes to the compiler toolchains and other packages coming from upstream, while the RHEL/CentOS machines will ensure that new features will not break Python on Arm on stable, slower moving distributions.
Is the code that you’re going to run 100% open source? If so, what is the URL or URLs where it is located?
What infrastructure (computing resources and network access) do you need? (see: https://www.packet.net/bare-metal/)?
The request is to provide a bare-metal AArch64 instance which will be maintained by Arm, myself and the Python language team. The machines for the Python CI are listening for github events and compiling Python with various configs for each commit. Since the CI system requires always-on machines, long-term support is required here.
Describe the continuous integration (CI) system in use or desired for this project.
The Python Programming Language utilizes the buildbot CI system.
You can see an overview of Python's infrastructure here: https://buildbot.python.org/all/#/
Currently there are only two arm buildbots, one for windows and one for Debian. I would like to expand the coverage to include more machines with different Linux flavors and configs.
Please state your contributions to the open source community and any other relevant initiatives
I am one of the maintainers for the Python Programming Language for Fedora and Red Hat Enterprise Linux. My work consists mainly on maintaining Python downstream for various architectures, including aarch64. I've worked on a variety of distribution wide initiatives, such as Python rebases, performance speedups, decoupling python2 from RHEL8/CentOS8 and others.
I've also contributed to various parts of CPython and other libraries and I work closely with Python's core developers, from which two of them are part of my team.
Additionally I already maintain four Intel buildbots for the Python CI system, running builds on Fedora development and stable versions, RHEL7 and RHEL8, with an s390x machine also coming soon.
Feel free to ask for any more information.