search

conda-forge / jaxlib-feedstock

A conda-smithy repository for jaxlib.
BSD 3-Clause "New" or "Revised" License
16 stars 24 forks source link

messed up 0.3.15 builds (should depend explicitly on abseil==*2022*) #132

Closed ngam closed 2 years ago

ngam commented 2 years ago

Solution to issue cannot be found in the documentation.

Issue

... python3.10/site-packages/jaxlib/xla_extension.so: undefined symbol: _ZN4absl12lts_202206235MutexD1Ev

looks like we broke the build again with abseil stuff...

Installed packages

n/a
0.3.15 cuda

Environment info

n/a
0.3.15 cuda
ngam commented 2 years ago

Nothing is inherently broken per se, but our pinning is really maddening and I cannot keep track anymore.

h-vetinari commented 2 years ago

Nothing is inherently broken per se, but our pinning is really maddening and I cannot keep track anymore.

You asked for grpc 1.46 with abseil 2021. Now you want 2022 after all?

The pinning is not so hard. Packages create a >= runtime dep for certain host deps, and some of those can only exist once in an environment, so all their dependencies need to have been built for that version.

More practically, if jaxlib was built successfully with grpc 1.46.3 and abseil 2021 before, then that is eminently still possible.

ngam commented 2 years ago

Now you want 2022 after all?

It is not me who wants it 🥲 (it is jaxlib: _ZN4absl12lts_202206235MutexD1Ev --- absl12lts_202206235). So what happened was that even thought abseil wasn't really used in the build, somehow very specific abseil was expected. I will be testing now if this will be resolved by the PRs.

More practically, if jaxlib was built successfully with grpc 1.46.3 and abseil 2021 before, then that is eminently still possible.

Yes 🤞

ngam commented 2 years ago

Nothing is inherently broken per se, but our pinning is really maddening and I cannot keep track anymore.

You asked for grpc 1.46 with abseil 2021. Now you want 2022 after all?

I think what I meant to say here was, I am struggling to figure out a way to capture all these issues. Do you know what I mean? I am not really sure what else we can do to prevent situations like this, except being overly vigilant. Crazily enough, I actually tested this on my M1 mac and things seemed fine, but the problems show up only on linux. This PR https://github.com/conda-forge/staged-recipes/pull/20096/checks was the first time I started worrying something was amiss (only on linux!)

h-vetinari commented 2 years ago

Yeah, I get that it's annoying, thanks for sticking with it. The underlying issue is not the complexity of the pinning, but untracked ABI exposure. If we knew didn't have gaps in the knowledge/metadata which ABIs a package is affected by, then the pinning+migrators would take care of it for us!

In this case, it might be a remnant from building against the "bad" grpc builds that didn't have the run-dep on abseil, which is an extra wriggle purely due to transition issues.

Obvious solutions aside from figuring out the ABI surface affecting Jax: build only with libs that worked before (and enforce appropriately), or move everything to the newest abseil homogeneously