Closed ryanvolz closed 3 years ago
We can add custom code to strip the links in the CDT builds. We'll need to implement support for per CDT build number bumps which has been on my list, but not done yet.
However, I'd first like the rest of @conda-forge/core to weigh in on this.
Using patchelf
to strip the dependencies sounds like a reasonable fix to me until there's builds we can access.
Defer to others as far as how to handle per CDT bumps as I'm not familiar with the current CDT machinery.
Defer to others as far as how to handle per CDT bumps as I'm not familiar with the current CDT machinery.
Yep. This task is on me.
Changing the CDT will not help as they are used only for building and this issue is about loading libraries.
Hrrrmmmmmm. Yeah you are right, but I am confused.
I managed a temporary workaround for this by adding the elfutils (for libdw.so.1) and libcap packages as host dependencies of gnuradio. In that case, loading libudev.so.1 finds the conda-provided versions of those libraries and so the conda version of libz is also used.
Maybe this isn't a workaround but the right solution? We should prefer conda packages to CDTs anyways.
There are 2 options
libz.so
to qtgui_python
before libudev
so that the conda zlib is loaded first and keep your fingers crossed that conda zlib is newer than system zlib.Thanks everyone! I'll think about making a proper conda package for libudev
. It didn't seem like it would be too crazy when I was checking out their build setup...
I am going to close this issue for now. I have merged changes to the code to allow individual build number bumps in the future.
This is a fun one. It started with this import error after building the
gnuradio
package forlinux-aarch64
and then attempting to do the same with thegnuradio-osmosdr
package:So importing some Qt stuff was loading
libpng16.so.16
, which in turn didn't find its requiredZLIB_1.2.9
(the version provided by the conda-forge package) because it was loading/lib64/libz.so.1
from the system instead of from the conda environment. Eventually I tried settingLD_DEBUG=libs
and saw the following relevant snippets. Early on we see:followed later by:
and then the error:
So the system path is coming from
libdw.so.1
. Searching the logs for that exposes the true culprit,libudev.so.1
:To summarize:
libudev.so.1
, provided by thesystemd-libs
CDT for cos7, pulls in both thelibdw.so.1
andlibcap.so.2
libraries from the system, which in turn leads to/lib64/libz.so.1
being used instead of the conda-forgelibz
and resulting in incompatibilities when other libraries expect the conda-forge version.That led me to the question of why this was happening with the CentOS 7 CDT and not earlier with the CentOS 6 CDT (
libudev-devel
). It looks like this is a casualty of bringinglibudev
into thesystemd
tree and build system; before that (cos6),libudev
didn't link tolibcap.so.2
andlibdw.so.1
at all, and after that it did. Moreover, the most recentsystemd
versions have corrected this so thatlibudev
no longer includes those links. Judging bysystemd
's build system source code through this transition, it seems likelibudev
was never intended to make those links as it doesn't actually use them, and they were included by mistake by virtue of being used elsewhere insystemd
. As far as I can tell, none of the newer fixed versions ofsystemd
have made it to CentOS 7. The currentsystemd-libs-cos7-*-219
CDT provideslibudev.so.1.6.2
, and from at leastlibudev.so.1.6.11
(fromsystemd-libs-239
, CentOS 8 rpm) the overlinking problem has been fixed. This is confirmed withldd
where those libraries are no longer present, and moreover we can see those unused dependencies (and more?):I managed a temporary workaround for this by adding the
elfutils
(forlibdw.so.1
) andlibcap
packages as host dependencies ofgnuradio
. In that case, loadinglibudev.so.1
finds the conda-provided versions of those libraries and so the conda version oflibz
is also used.I think it is likely that others will encounter this problem whenever conda-forge transitions to CentOS 7, so I'd like to see if there is a better solution. I don't imagine that using a CentOS 8 RPM that has a fixed
libudev
would work or be the right solution. Maybe it would be possible to replace thelibudev
portion of thesystemd-libs
CDT with a proper conda-forge package. The best solution might be to modify the currentlibudev.so.1.6.2
during the CDT packaging and strip the unused libraries. Something like:I could probably come up with a PR that does that, if that would be acceptable and assuming it would work. Let me know if I've missed anything and what the best path forward might be. Thanks!