search

pypa / manylinux

Python wheels that work on any linux (almost)
MIT License
1.44k stars 218 forks source link

Tracking issue for manylinux_2_28 image #1282

Closed mattip closed 2 years ago

mattip commented 2 years ago

Due to issue #1012, the manylinux_2_24 image proposed as the first PEP600 compliant image did not take off. I think we should offer an image based on manylinux_2_28.

The big question is what base image to use. We can choose between C8S (centos:stream8), AlmaLinux, RockyLinux, or UBI. Personally I would prefer an image based entirely on open source software. There is a thread about this on discuss.python.org

base image EOL ¹ gcc-toolset-11 aarch64 ppc64le s390x bug fix availability
CentOS Stream 8 2024-05-31 ² yes yes yes planned ³ 1st
UBI 8 2029-05-31 NO yes yes yes 2nd
AlmaLinux 8 2029-05-31 yes yes yes soon ⁴ 3rd
RockyLinux 8 2029-05-31 yes yes planned ⁵ planned ⁵ 3rd

¹ including Maintenance Support, https://access.redhat.com/support/policy/updates/errata/ ² https://wiki.centos.org/About/Product ³ https://github.com/pypa/manylinux/issues/1282#issuecomment-1038304351 ⁴ https://almalinux.discourse.group/t/plans-for-s390x/875 ⁵ after RockyLinux 9 is out, https://forums.rockylinux.org/t/rocky-linux-8-4-available-now/3015/48

The checklist is short, since wheel and pip, which both use packaging, and warehouse do not require additional PRs.

Publisher-side Support:

Documentation:

Additional projects to check for support for pernnial wheels (left over from gh-542, perhaps these are already done?

duburcqa commented 2 years ago

Here is what I'm doing personally:

PYTHON_VERSION="cp39"
pythonLocation=$(find /opt/python -maxdepth 1 -name "${PYTHON_VERSION}-*" -print -quit)
export PATH="${pythonLocation}/bin:${PATH}"
h-vetinari commented 2 years ago

Regarding the tracker on top, cibuildwheel support was merged and released recently, same for maturin already a while ago, and multibuild doesn't seem to require (non-doc) changes?

h-vetinari commented 2 years ago

dockcross now has 2_28 as well: https://github.com/dockcross/dockcross/pull/712

h-vetinari commented 1 year ago

@mayeut: UBI 8:

  • pros:
    • [...]
  • cons:
    • gcc-toolset-11 not available
    • most devel packages are not available.
    • only provides a subset of the packages the other ones provide

@carlwgeorge: Per the UBI FAQ #35, gcc-toolset-11 and anything else required can be requested by opening a bugzilla. I can't make any promises, but if it would make UBI the preferred choice for this, I suspect it would be viewed quite favorably. cc: @fatherlinux

@fatherlinux: I'll look into that veraion of gcc-toolkit :-)

Sorry for necroposting, but just wanted to check how the gcc-toolkit-on-UBI situation looks like[^1]. This is less relevant now for UBI 8 (where we've gone with Almalinux already anyway), but it would be relevant for UBI 9 some time down the road.

[^1]: I had a look through the RHEL docs but couldn't find anything about the gcc-toolkit/devtoolset version in UBI.

carlwgeorge commented 1 year ago

UBI 9 currently has GCC 11.3 as the default, and an optional SCL for GCC 12.2.

[root@b4af68881a12 ~]# dnf repoquery --quiet --nvr gcc gcc-toolset-12-gcc
gcc-11.3.1-4.3.el9
gcc-toolset-12-gcc-12.2.1-7.4.el9

UBI 8 currently has GCC 8.5 as the default, and an optional SCL for GCC 12.1.

[root@9655f383e722 ~]# dnf repoquery --quiet --nvr gcc gcc-toolset-12-gcc
gcc-8.5.0-16.el8_7
gcc-toolset-12-gcc-12.1.1-3.4.el8_7
h-vetinari commented 1 year ago

Ah, that's great news, thanks! At the time this issue was discussed last year, the gcc-toolset didn't yet exist on rhubi, AFAIR, very happy to hear this got added in the meantime!

That IMO makes rhubi 9 a very good candidate for 2_34, and rhubi 8 a good fallback for 2_28 in case anything with the alma project were to go so spectacularly wrong that we'd have to switch.

h-vetinari commented 1 year ago

[that makes] rhubi 8 a good fallback for 2_28 in case anything with the alma project were to go so spectacularly wrong that we'd have to switch.

You know, I really didn't intend to jinx this, but it seems that RHEL is putting the screws on the derivative distros... https://almalinux.org/blog/impact-of-rhel-changes/

If Alma is forced to change its model (looks possible ATM), we should start looking into rhubi I guess...

snnn commented 1 year ago

ubi8 only has gcc 8 and gcc 12. It doesn't have gcc 11 or 10 or 9. And CUDA 11.x isn't compatible with gcc 12.

h-vetinari commented 1 year ago

The good thing is that it's looking like Alma will be able to continue to provide support beyond CentOS' EOL, only they have given up bug-for-bug compatibility (but will still stay ABI-compatible, which is what counts). So it looks like there won't be any action necessary for manylinux_2_28.

snnn commented 6 months ago

From security point of view, I have many concerns on UBI. The table on the top isn't accurate. Though it says you can get security updates for UBI8 till 2029-05-31, the real situation is more complicated. First, we need to get clear what UBI is. UBI docker images are customized RHELs. For example, if you need to use python, you can get a UBI docker image with python from Red Hat, and you need to keep in mind that:

  1. RHEL is not free. If you did not pay, you cannot get access their security updates.
  2. Only Red Hat can make UBI images. You can build your images based on these UBI images, but you should not add new RPM packages from Red Hat's repos.

The images you get from Red Hat are well maintained in the sense that if the corresponding RHEL has published a security update for a software that is contained in a UBI image (that was made by Red Hat), the security update will be included in an updated UBI image within 6 weeks. However, you cannot get the updated RPM package from Red Hat for free. If you run "dnf repolist" command in a UBI image, you will see things like:

repo id                                       repo name
ubi-8-appstream-rpms                          Red Hat Universal Base Image 8 (RPMs) - AppStream
ubi-8-baseos-rpms                             Red Hat Universal Base Image 8 (RPMs) - BaseOS
ubi-8-codeready-builder-rpms                  Red Hat Universal Base Image 8 (RPMs) - CodeReady Builder

Unlike RHEL, it doesn't have any "update" repos. If you query the package version of the gmp package in a UBI8 image, you will see:

# rpm -qa |grep gmp
gmp-6.1.2-11.el8_8.1.x86_64

The version contains the necessary updates for Security Advisory RHSA-2024:1102, which is good. However, you cannot find the updated RPM package from the three repos above. It means anything installed by youself with "dnf install" command will not get enough security updates. This manylinux repo uses the command a lot. That's my concern. To remediate the problem, we need to ask Red Hat to build and publish the manylinux images, which means we need to let them take over this project. I will be more concerned if we take that approach.

Therefore, from security point of view, UBI is not a viable solution here.

carlwgeorge commented 5 months ago

The table on the top isn't accurate. Though it says you can get security updates for UBI8 till 2029-05-31, the real situation is more complicated.

The table is accurate. UBI8 has the same EOL date as RHEL8 (2029-05-31).

First, we need to get clear what UBI is. UBI docker images are customized RHELs.

It's not "customized RHEL", it's a subset of RHEL. The packages in UBI repos are the exact same as the corresponding packages in RHEL. When a RHEL package is updated, it goes out the the RHEL repos, and if it's on the list of UBI packages, it goes out to the UBI repos at the same time.

RHEL is not free. If you did not pay, you cannot get access their security updates.

The subset of RHEL packages that are included in UBI are free and redistributable. That's the whole point of UBI. That set of packages get all the same security updates that RHEL itself gets.

You can build your images based on these UBI images, but you should not add new RPM packages from Red Hat's repos.

You can absolutely add packages from the UBI repos to create custom UBI-based images. What you can't do is add a package from a RHEL subscription (i.e. outside the UBI content set) to a UBI image and redistribute that.

The images you get from Red Hat are well maintained in the sense that if the corresponding RHEL has published a security update for a software that is contained in a UBI image (that was made by Red Hat), the security update will be included in an updated UBI image within 6 weeks. However, you cannot get the updated RPM package from Red Hat for free.

Yes, you can. All of the packages in the UBI images, as well as additional packages, are in the UBI repos and you get updates to those packages for free.

If you run "dnf repolist" command in a UBI image, you will see things like:

repo id                                       repo name
ubi-8-appstream-rpms                          Red Hat Universal Base Image 8 (RPMs) - AppStream
ubi-8-baseos-rpms                             Red Hat Universal Base Image 8 (RPMs) - BaseOS
ubi-8-codeready-builder-rpms                  Red Hat Universal Base Image 8 (RPMs) - CodeReady Builder

Unlike RHEL, it doesn't have any "update" repos.

RHEL doesn't have "update" repos either.

If you query the package version of the gmp package in a UBI8 image, you will see:

# rpm -qa |grep gmp
gmp-6.1.2-11.el8_8.1.x86_64

The version contains the necessary updates for Security Advisory RHSA-2024:1102, which is good. However, you cannot find the updated RPM package from the three repos above.

This one is actually a bit strange, but not for the reason you are suggesting. To explain, allow me to reference the RHEL 8 Planning Guide diagram. You can see that RHEL 8 has minor versions, and some of those minor versions have lifecycle extensions. The current versions are 8.9, 8.8 EUS, 8.6 EUS, 8.4 SAP, and 8.2 SAP. What may not be obvious in this chart is that CentOS Stream 8 is effectively the 8.10 version, which hasn't been released as RHEL 8.10 yet. When a bug is being fixed, the RHEL maintainer chooses how many branches (minor versions) to apply the fix to. Some bugs are only fixed in CentOS Stream, later showing up in the next RHEL minor version. Some bugs are fixed in CentOS Stream and the current RHEL minor version. Some bugs are fixed in other combinations of next, current, EUS, and SAP versions. In this case, the CVE in question has been fixed in these versions of RHEL 8:

Notably, this has not been fixed in RHEL 8.9 (the current version), which is why an update for it is not available in the UBI 8 repos. However, somehow gmp-6.1.2-11.el8_8.1 snuck into the UBI 8 image, which is how you saw it with rpm -qa. This is not something being held back in UBI per se, it's something that the maintainer chose to fix in 8.6 EUS, 8.8 EUS, and the upcoming 8.10 only, and UBI 8 appears to have accidentally gotten it baked into the image while it isn't available in the UBI 8 or RHEL 8.9 repos. When you run a UBI 8 image on a subscribed RHEL system, its repos transition to the full RHEL content set, and even in that scenario it doesn't yet see an update to gmp that fixes that CVE because the fix isn't in RHEL 8.9.

All that said, while this is an odd situation, it is not caused by a security update being unavailable in the UBI repos, as you suggested. It will also be resolved this month or next when RHEL 8.10 is released.

It means anything installed by youself with "dnf install" command will not get enough security updates.

That is false. Lets say you run dnf install gnutls in a UBI 8 container. Right now that gives you gnutls-3.6.16-8.el8_9.3, which is an update that was published to RHEL 8.9 and UBI 8 on 2024-04-11.

Therefore, from security point of view, UBI is not a viable solution here.

I'm happy to engage here to help answer questions about UBI, RHEL, and CentOS Stream, but I'm going to need to you avoid passing off assumptions as fact, and drawing conclusions from that. UBI is a perfectly viable solution, from a security point of view or otherwise, as long as all the necessary packages are included in that UBI content set. If there is anything missing I'm happy to help advocate inside Red Hat for it to be added.

snnn commented 5 months ago

Thanks for the explanation and being supportive. So the security patch for gmp landed in the image first(because of a mistake), while the security patches for expat, gnutls, tzdata and libsemanage landed in the dnf repos first(which is normal). I apologize for my mistake that I only saw a piece of the whole picture.