search

cschwan / sage-on-gentoo

(Unofficial) Gentoo Overlay for Sage- and Sage-related ebuilds
84 stars 26 forks source link

RUNPATH still present after removal of .la files #609

Closed strogdon closed 2 years ago

strogdon commented 3 years ago

I still have RUNPATH in libgiac.so after rebuilding nauty and giac:

# eix -I nauty && eix -I giac
[I] sci-mathematics/nauty
     Available versions:  2.5.9 (~)2.6.1[1] 2.6.7^t (~)2.6.7-r1^t[1] {test}
     Installed versions:  2.6.7-r1^t[1](07:10:03 PM 10/21/2020)(-test)
     Homepage:            http://pallini.di.uniroma1.it/
     Description:         Computing automorphism groups of graphs and digraphs

[1] "sage-on-gentoo" /var/lib/layman/sage-on-gentoo
[I] sci-mathematics/giac
     Available versions:  ~1.1.0[2] (~)1.5.0.87[1] ~1.6.0.25^t[1] {ao doc +ecm examples fltk gc +glpk l10n_ static-libs test L10N="el en es fr pt"}
     Installed versions:  1.5.0.87[1](07:17:41 PM 10/21/2020)(ecm glpk -ao -doc -examples -fltk -gc -static-libs L10N="en fr -el -es -pt")
     Homepage:            https://www-fourier.ujf-grenoble.fr/~parisse/giac.html
     Description:         A free C++ CAS (Computer Algebra System) library and its interfaces

[1] "sage-on-gentoo" /var/lib/layman/sage-on-gentoo
[2] "science" /var/lib/layman/science
# objdump -x /usr/lib64/libgiac.so | grep RUNPATH
  RUNPATH              //usr/lib64

with of course linking to libblas.so.3

# ldd -r /usr/lib64/libgiac.so | grep blas
        libblas.so.3 => //usr/lib64/libblas.so.3 (0x00007ff6c05f6000)
        libcblas.so.3 => /usr/lib64/blas/openblas/libcblas.so.3 (0x00007ff6bdc92000)
        libopenblas.so.0 => /usr/lib64/libopenblas.so.0 (0x00007ff6bb4c4000)

Am I correctly doing this?

kiwifb commented 3 years ago

You are doing this right as far as I can tell. If you have the build log for giac, I would like to see line linking libgiac.so. A grep for that particular string should only return a few lines.

strogdon commented 3 years ago

This is what I have from vanilla

$ ldd -r local/lib/libgiac.so
        linux-vdso.so.1 (0x00007ffd5cb57000)
        libntl.so.43 => /usr/lib64/libntl.so.43 (0x00007f1a33815000)
        libpari-gmp-tls.so.6 => /usr/lib64/libpari-gmp-tls.so.6 (0x00007f1a32d10000)
        libgsl.so.23 => /usr/lib64/libgsl.so.23 (0x00007f1a328a3000)
        libopenblas.so.0 => /usr/lib64/libopenblas.so.0 (0x00007f1a319b1000)
        libnauty.so.2 => /usr/lib64/libnauty.so.2 (0x00007f1a31744000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f1a31525000)
        libcurl.so.4 => /usr/lib64/libcurl.so.4 (0x00007f1a312ac000)
        libglpk.so.40 => /usr/lib64/libglpk.so.40 (0x00007f1a30fd3000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f1a30dcf000)
        libecm.so.1 => /usr/lib64/libecm.so.1 (0x00007f1a30b55000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f1a3094d000)
        libmpfi.so.0 => /usr/lib64/libmpfi.so.0 (0x00007f1a30736000)
        libmpfr.so.6 => /usr/lib64/libmpfr.so.6 (0x00007f1a3028c000)
        libgmp.so.10 => /usr/lib64/libgmp.so.10 (0x00007f1a30014000)
        libstdc++.so.6 => /usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/libstdc++.so.6 (0x00007f1a2fb9d000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f1a2f869000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f1a2f4b0000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f1a35442000)
        libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/libgcc_s.so.1 (0x00007f1a2f298000)
        libgf2x.so.1 => /usr/lib64/libgf2x.so.1 (0x00007f1a2f081000)
        libcblas.so.3 => /usr/lib64/blas/openblas/libcblas.so.3 (0x00007f1a2ee45000)
        libgfortran.so.5 => /usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/libgfortran.so.5 (0x00007f1a2e9ae000)
        libgomp.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/libgomp.so.1 (0x00007f1a2e779000)
        libnghttp2.so.14 => /usr/lib64/libnghttp2.so.14 (0x00007f1a2e551000)
        libssl.so.1.1 => /usr/lib64/libssl.so.1.1 (0x00007f1a2e2c1000)
        libcrypto.so.1.1 => /usr/lib64/libcrypto.so.1.1 (0x00007f1a2de06000)
        libzstd.so.1 => /usr/lib64/libzstd.so.1 (0x00007f1a2db63000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f1a2d94c000)
        libcolamd.so.0 => /usr/lib64/libcolamd.so.0 (0x00007f1a2d745000)
        libamd.so.0 => /usr/lib64/libamd.so.0 (0x00007f1a2d53c000)
        libquadmath.so.0 => /usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/libquadmath.so.0 (0x00007f1a2d2f5000)
        libsuitesparseconfig.so.0 => /usr/lib64/libsuitesparseconfig.so.0 (0x00007f1a2d0f2000)
kiwifb commented 3 years ago

That built properly but the giac spkg may have looked for openblas directly.

kiwifb commented 3 years ago

As talked about in private email, at least gmp-ecm also needs to be fixed.

strogdon commented 3 years ago

Removed the gmp-ecm provided .la file, /usr/lib64/libecm.la, rebuilt giac and things seem fine. There is no RUNPATH in libgiac.so and linking for blas looks correct

ldd -r /usr/lib64/libgiac.so | grep blas
        liblapack.so.3 => /usr/lib64/lapack/openblas/liblapack.so.3 (0x00007f4e06c7f000)
        libblas.so.3 => /usr/lib64/blas/openblas/libblas.so.3 (0x00007f4e06a48000)
        libcblas.so.3 => /usr/lib64/blas/openblas/libcblas.so.3 (0x00007f4e040e4000)
        libopenblas.so.0 => /usr/lib64/libopenblas.so.0 (0x00007f4e031f2000)
strogdon commented 3 years ago

Doing the same in Prefix I have some aberrant behavior. There is still linking to both blas and lapack reference

ldd -r ~/usr/lib/libgiac.so | grep blas
        libblas.so.3 => /storage/strogdon/gentoo-rap/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../lib64/libblas.so.3 (0x00007fbbf56f8000)
        libcblas.so.3 => /storage/strogdon/gentoo-rap/usr/lib64/blas/openblas/libcblas.so.3 (0x00007fbbf2c41000)
        libopenblas.so.0 => /storage/strogdon/gentoo-rap/usr/lib64/libopenblas.so.0 (0x00007fbbf0357000)

and

ldd -r ~/usr/lib/libgiac.so | grep lapack
        liblapack.so.3 => /storage/strogdon/gentoo-rap/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../lib64/liblapack.so.3 (0x00007f0be9da7000)

There is also a RUNPATH entry

$ objdump -x ~/usr/lib/libgiac.so | grep RUNPATH
  RUNPATH              /storage/strogdon/gentoo-rap/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../lib64

which may be difficult to accommodate.

strogdon commented 3 years ago

Prefix issue resolved. I had an old version of mpfi installed (mpfi-1.5.2). Upgraded to mpfi-1.5.3 and things look good

$ ldd -r ~/usr/lib/libgiac.so | grep blas
        liblapack.so.3 => /storage/strogdon/gentoo-rap/usr/lib64/lapack/openblas/liblapack.so.3 (0x00007fe5101a8000)
        libblas.so.3 => /storage/strogdon/gentoo-rap/usr/lib64/blas/openblas/libblas.so.3 (0x00007fe50ff71000)
        libcblas.so.3 => /storage/strogdon/gentoo-rap/usr/lib64/blas/openblas/libcblas.so.3 (0x00007fe50d4ba000)
        libopenblas.so.0 => /storage/strogdon/gentoo-rap/usr/lib64/libopenblas.so.0 (0x00007fe50c5cd000)

And there is no RUNPATH in libgiac.so.

strogdon commented 3 years ago

Somewhat related, at least relative to linking. I see on both gentoo and prefix

$ ls -alR /etc/env.d/alternatives/
/etc/env.d/alternatives/:
total 20
drwxr-xr-x 5 root root 4096 Aug 25  2017 .
drwxr-xr-x 8 root root 4096 Nov  5 20:39 ..
drwxr-xr-x 2 root root 4096 Feb  7  2020 blas
drwxr-xr-x 2 root root 4096 Feb  7  2020 cblas
drwxr-xr-x 2 root root 4096 Feb  7  2020 lapack

/etc/env.d/alternatives/blas:
total 12
drwxr-xr-x 2 root root 4096 Feb  7  2020 .
drwxr-xr-x 5 root root 4096 Aug 25  2017 ..
lrwxrwxrwx 1 root root   15 Mar 29  2019 _current -> openblas-openmp
-rw-r--r-- 1 root root   29 Mar 29  2019 _current_list

/etc/env.d/alternatives/cblas:
total 12
drwxr-xr-x 2 root root 4096 Feb  7  2020 .
drwxr-xr-x 5 root root 4096 Aug 25  2017 ..
lrwxrwxrwx 1 root root   15 Mar 29  2019 _current -> openblas-openmp
-rw-r--r-- 1 root root   30 Mar 29  2019 _current_list

/etc/env.d/alternatives/lapack:
total 12
drwxr-xr-x 2 root root 4096 Feb  7  2020 .
drwxr-xr-x 5 root root 4096 Aug 25  2017 ..
lrwxrwxrwx 1 root root    9 Aug 25  2017 _current -> reference
-rw-r--r-- 1 root root   31 Aug 25  2017 _current_list

All of the _current symlinks are stale and some are very old. I don't know which packages own what? What, if anything, can be cleared? Found this while investigating weird complaints from ldconfig.

>>> Installing (1 of 1) sys-auth/pambase-20201103::gentoo
/sbin/ldconfig: /usr/lib64/blas/openblas/libblas.so.3 is not a symbolic link

which may be unrelated.

kiwifb commented 3 years ago

The stuff in /etc/env.d/alternatives is from the old framework we had in the science overlay, I am fairly sure - you should be able to delete it without issues. I got the last one too about libblas not being a symlink and yes it should be one to whatever library openblas installs. It is OK if it is an identical copy, which I haven't really checked.

strogdon commented 3 years ago

Which cblas should giac be using? From comment above I had cblas linked to libs provided by openblas. But now (I don't know what has changed) I have cblas linked to libs provide by reference

$ ldd -r /usr/lib64/libgiac.so | grep blas
        liblapack.so.3 => /usr/lib64/lapack/openblas/liblapack.so.3 (0x00007f0fa8cbb000)
        libblas.so.3 => /usr/lib64/blas/openblas/libblas.so.3 (0x00007f0fa8a56000)
        libcblas.so.3 => /usr/lib64/libcblas.so.3 (0x00007f0fa6114000)
        libopenblas.so.0 => /usr/lib64/libopenblas.so.0 (0x00007f0fa51d3000)
kiwifb commented 3 years ago

It should be using whatever you selected for blas and cblas. It gets blas through linking to lapack and cblas through linking to gsl. A more interesting read would be from readelf -d.

strogdon commented 3 years ago 
$ readelf -d /usr/lib64/libgiac.so

Dynamic section at offset 0x12b6d28 contains 43 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libntl.so.43]
 0x0000000000000001 (NEEDED)             Shared library: [libpari-gmp-tls.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgsl.so.23]
 0x0000000000000001 (NEEDED)             Shared library: [liblapack.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libblas.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libnauty.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libcurl.so.4]
 0x0000000000000001 (NEEDED)             Shared library: [libglpk.so.40]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libecm.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libmpfi.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libmpfr.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgmp.so.10]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x000000000000000e (SONAME)             Library soname: [libgiac.so.0]
 0x000000000000000c (INIT)               0x1b38f0
 0x000000000000000d (FINI)               0xfdc8d0
 0x0000000000000019 (INIT_ARRAY)         0x1488098
 0x000000000000001b (INIT_ARRAYSZ)       448 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x1488258
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x000000006ffffef5 (GNU_HASH)           0x1c8
 0x0000000000000005 (STRTAB)             0x587e8
 0x0000000000000006 (SYMTAB)             0x10758
 0x000000000000000a (STRSZ)              653393 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000003 (PLTGOT)             0x14be000
 0x0000000000000002 (PLTRELSZ)           149040 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x18f2c0
 0x0000000000000007 (RELA)               0xfe288
 0x0000000000000008 (RELASZ)             593976 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x000000006ffffffe (VERNEED)            0xfe048
 0x000000006fffffff (VERNEEDNUM)         8
 0x000000006ffffff0 (VERSYM)             0xf803a
 0x000000006ffffff9 (RELACOUNT)          20362
 0x0000000000000000 (NULL)               0x0

I don't see any cblas unless it's coming from gsl. cblas is linked to reference in gsl, but I haven't rebuilt gsl recently.

kiwifb commented 3 years ago

You can see from that, that you get blas and lapack from direct linking and that blas and lapack functions are being used. You only get cblas indirectly from gsl. ldd resolves all libraries, so if any of the libraries listed above needs another one, it will shown by ldd. And this is the kind of situation where you could end up with several versions of (c)blas showing up depending on how you linked things.

strogdon commented 3 years ago

So why in comment did I have linking of cblas to openblas ?

kiwifb commented 3 years ago

Darn, I have to read those things carefully. Something linked directly to /usr/lib64/libcblas.so - which is reference. Something links directly to it. It looks like gsl is weird

fbissey@moonloop ~ $ ldd -r /usr/lib64/libgsl.so
    linux-vdso.so.1 (0x00007ffcb79c6000)
    libcblas.so.3 => /usr/lib64/libcblas.so.3 (0x00007fefc9fa5000)
    libm.so.6 => /lib64/libm.so.6 (0x00007fefc9e70000)
    libc.so.6 => /lib64/libc.so.6 (0x00007fefc9cb5000)
    libblas.so.3 => /usr/lib64/blas/openblas/libblas.so.3 (0x00007fefc9c54000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fefca25e000)
    libopenblas.so.0 => /usr/lib64/libopenblas.so.0 (0x00007fefc8f04000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fefc8ee4000)
    libgfortran.so.5 => /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/libgfortran.so.5 (0x00007fefc8c20000)
    libquadmath.so.0 => /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/libquadmath.so.0 (0x00007fefc8bd6000)
    libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/libgcc_s.so.1 (0x00007fefc8bbc000)
fbissey@moonloop ~ $ readelf -d /usr/lib64/libgsl.so

Dynamic section at offset 0x259a68 contains 27 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libcblas.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000e (SONAME)             Library soname: [libgsl.so.23]
 0x000000000000000c (INIT)               0x57000
kiwifb commented 3 years ago

I know. that's because 

fbissey@moonloop ~ $ ldd -r /usr/lib64/libcblas.so.3
    linux-vdso.so.1 (0x00007ffe5cfbe000)
    libblas.so.3 => /usr/lib64/blas/openblas/libblas.so.3 (0x00007f8c64177000)
    libc.so.6 => /lib64/libc.so.6 (0x00007f8c63fbc000)
    libopenblas.so.0 => /usr/lib64/libopenblas.so.0 (0x00007f8c6326c000)
    libm.so.6 => /lib64/libm.so.6 (0x00007f8c63137000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f8c64225000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f8c63117000)
    libgfortran.so.5 => /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/libgfortran.so.5 (0x00007f8c62e55000)
    libquadmath.so.0 => /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/libquadmath.so.0 (0x00007f8c62e09000)
    libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/libgcc_s.so.1 (0x00007f8c62def000)

reference libcblas links to blas - which is for us openblas. The mystery is why libcblas is resolved as /usr/lib64/libcblas rather than /usr/lib64/blas/openblas/libcblas.

strogdon commented 3 years ago

Is is because of the /sbin/ldconfig: /usr/lib64/blas/openblas/libblas.so.3 is not a symbolic link. I rebuilt gsl and got the symlink issue.

kiwifb commented 3 years ago

OK that is wrong and is probably one of the reason we have trouble

fbissey@moonloop ~ $ readelf -d /usr/lib64/blas/openblas/libcblas.so.3 

Dynamic section at offset 0x38c80 contains 27 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libopenblas.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000e (SONAME)             Library soname: [libblas.so.3]

the soname for the openblas cblas library is libblas.so.3 ???

kiwifb commented 3 years ago

I have to attend to RL for a bit.

kiwifb commented 3 years ago

OK the wrong soname is really the problem. It prevents proper name resolution and was introduced in openblas-0.3.12. In openblas-0.3.12-shared-blas-lapack.patch we have

+libcblas.so.3: $(CSBLAS1OBJS) $(CSBLAS2OBJS) $(CSBLAS3OBJS) $(CDBLAS1OBJS) $(CDBLAS2OBJS) $(CDBLAS3OBJS) $(CCBLAS1OBJS) $(CCBLAS2OBJS) $(CCBLAS3OBJS) $(CZBLAS1OBJS) $(CZBLAS2OBJS) $(CZBLAS3OB>
+       $(CC) $(LDFLAGS) -shared -o $@ $^ -Wl,-soname,libblas.so.3 -L.. -lopenblas $(EXTRALIB)

and if you correct the second line to have the right name, the complaints about the symlink stop and libcblas.so.3 gets resolved properly. This is an official bug in openblas-0.3.12 in Gentoo.

strogdon commented 3 years ago

Yes, openblas-0.3.10 is fine

readelf -d /usr/lib64/blas/openblas/libcblas.so

Dynamic section at offset 0x3ac60 contains 28 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libopenblas.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgomp.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000e (SONAME)             Library soname: [libcblas.so.3]
kiwifb commented 3 years ago

Currently filling the bug in bugzilla. Stay tuned.

kiwifb commented 3 years ago

This is now https://bugs.gentoo.org/753437

kiwifb commented 3 years ago

And that bug is now resolved. One less linking issue.

strogdon commented 3 years ago

I noticed that I have libgomp linked into my build of openblas's libcblas and you don't. I presume because I have openmp enabled. Are there any Sage components for which openmp should not be enabled? I don't see on my system that openmp is explicitly enabled in the build of openblas but it is built that way? I do have openmp enabled in fflas-ffpack and explicitly disabled in linbox and gmp-ecm.

kiwifb commented 3 years ago

I had some issue at some point. I decided to just do plain threading. There are issue with doing openmp openblas from an openmp enabled application safely. It may have improved since I last looked.