ComputeCanada / wheels_builder

10 stars 2 forks source link

Use gentoo2020/2023 for SIGILL-causing wheels #100

Closed bartoldeman closed 1 year ago

bartoldeman commented 1 year ago

I think we also need that here, for debugpy.

mboisson commented 1 year ago

Wouldn't it be only if EBROOTGENTOO is 2023 ? (i.e. 2020 would be compatible with 2023 as it is built with a lower arch, but the opposite isn't true)

bartoldeman commented 1 year ago

I have so far avoided sharing non-generic wheels between 2020 and 2023, as soon as there is a non-glibc dep there could be issues (if there is libabc.so.0 in 2020 but libabc.so.1 in 2023). So I'd rather keep it more simple and just have generic and arch-specific for 2023, no other borrowing from 2020.

Though the difference between gentoo and gentoo2020 in the wheelhouse becomes a bit murky then, I'm not sure what's the point of the two folders then, other than clarity (for gentoo2020) and backwards compatibility for gentoo

mboisson commented 1 year ago

I assumed that there was still significant overlap between libraries in 2020 and 2023. Is that no the case ?

bartoldeman commented 1 year ago

There is significant overlap. It would just need to be checked for. The other thing is that that there is a price to pay: a GCC-12.3.0 recompiled wheel may be faster and take advantage of new glibc functionality than an older GCC-9.3.0 wheel (even if that wheel still works). For which exact wheels, I have no idea, but it seems that as the vast majority of wheels is already completely generic, we may as well do rebuilds for the arch specific ones, and save ourselves the compatibility checks, to get a simpler setup.

bartoldeman commented 1 year ago

I think I was also a bit confused about gentoo vs gentoo2020: wheel_architecture.sh has this:

        elif [[ $WORKS_ON_GENTOO2020 -eq 1 && $WORKS_ON_GENTOO2023 -eq 1 ]]; then
                COMPATIBILITY_LAYER="gentoo"
        elif [[ $WORKS_ON_GENTOO2023 -eq 1 ]]; then
                COMPATIBILITY_LAYER="gentoo2023"
        elif [[ $WORKS_ON_GENTOO2020 -eq 1 ]]; then
                COMPATIBILITY_LAYER="gentoo2020"

but in the gentoo2020 there are only newer wheels, I supposed that when this was merged you meant to move the older gentoo wheels to gentoo2020, depending on what wheel_architecture.sh tells you about them?

mboisson commented 1 year ago

I supposed that when this was merged you meant to move the older gentoo wheels to gentoo2020, depending on what wheel_architecture.sh tells you about them?

Yes, that was the next step when we are ready. I expect most wheels to end up in gentoo rather than gentoo2020, but those that have missing libraries in 2023 would end up in gentoo2020

bartoldeman commented 1 year ago

I made the change to apply only if EBROOTGENTOO is 2023 as requested. It then bumps both generic and gentoo to gentoo2023 if --arch is specified.

mboisson commented 1 year ago

lgtm