Open eserte opened 2 years ago
Thanks for the report! I'd really like to crack this properly.
This really doesn't look like it got added recently to LAPACK (no mention in release notes going back to 3.7.0 in Dec 2016), but - would you know which version of LAPACK you have on your various systems? Would you also be able to use nm
on the LAPACK libraries in question to see if they have these, with and/or without a leading or trailing _
?
sgesdd
(apparently missing)sgetrf
(another one that's used)Also, what ExtUtils::F77 returns for trail_
?
I'm seeing the library find all relevant symbols and pass tests on Ubuntu, MacOS, and Strawberry Perl.
It seems that the problem happens if liblapack-dev is not installed. In the end the *.so files were built without lapack linked in. I installed now liblapack-dev on my debian:11 smoker (but only there), and at least one perl there has a passing test suite. Builds for other perls are running right now.
Aha! Yes, one interesting artifact of the current setup is that P:LA does all its own declarations of (the C-ish analogue of the Fortran functions), so the lack of a -dev
wouldn't hold it back. What's a bit mysterious is that with the Devel::CheckLib stuff, that's actually a test of finding the relevant libraries (it tries to run ilaenv
). I don't understand how that passed. Could this be that my attempt at defeating the optimiser failed?https://github.com/PDLPorters/pdl-linearalgebra/blob/33a30c69bb9e15ff006e5f4293ba07544ba401d3/Makefile.PL#L39
In any case, do the builds work now? I'd like to avoid what seems to me a false positive before (of the presence of LAPACK), so any insights you can offer will be incredibly helpful for the near-future Alien::LAPACK :-)
Not sure if this helps or not, but here's what I did to troubleshoot on CentOS 7 for this case:
# find files in packages that contain the missing function:
rpm -ql `rpm -qa |grep openblas` `rpm -qa |grep atlas` | xargs grep sgesdd_ | grep matches | cut -f3 -d' ' | sort -u > /tmp/libs
# grep the missing functions from the matching libs:
for a in `cat /tmp/libs`; do readelf -Ws --dyn-syms $a | while read b ; do echo "$a: $b"; done;done | grep -P '[^_](dgebal|sgesdd)_$'
You could do something similar on Ubuntu/Debian with dpkg -L <pkgname>
instead of rpm -ql <pkgname>
for listing files.
The rpm out looks like this:
/usr/lib64/atlas/libsatlas.so: 5948: 00000000001db4c0 2944 FUNC GLOBAL DEFAULT 11 dgebal_
/usr/lib64/atlas/libsatlas.so: 6136: 000000000032b8a0 21223 FUNC GLOBAL DEFAULT 11 sgesdd_
/usr/lib64/atlas/libsatlas.so.3: 5948: 00000000001db4c0 2944 FUNC GLOBAL DEFAULT 11 dgebal_
/usr/lib64/atlas/libsatlas.so.3: 6136: 000000000032b8a0 21223 FUNC GLOBAL DEFAULT 11 sgesdd_
/usr/lib64/atlas/libsatlas.so.3.10: 5948: 00000000001db4c0 2944 FUNC GLOBAL DEFAULT 11 dgebal_
/usr/lib64/atlas/libsatlas.so.3.10: 6136: 000000000032b8a0 21223 FUNC GLOBAL DEFAULT 11 sgesdd_
/usr/lib64/atlas/libtatlas.so: 5975: 00000000001f4480 2944 FUNC GLOBAL DEFAULT 11 dgebal_
/usr/lib64/atlas/libtatlas.so: 6172: 0000000000344860 21223 FUNC GLOBAL DEFAULT 11 sgesdd_
/usr/lib64/atlas/libtatlas.so.3: 5975: 00000000001f4480 2944 FUNC GLOBAL DEFAULT 11 dgebal_
/usr/lib64/atlas/libtatlas.so.3: 6172: 0000000000344860 21223 FUNC GLOBAL DEFAULT 11 sgesdd_
/usr/lib64/atlas/libtatlas.so.3.10: 5975: 00000000001f4480 2944 FUNC GLOBAL DEFAULT 11 dgebal_
/usr/lib64/atlas/libtatlas.so.3.10: 6172: 0000000000344860 21223 FUNC GLOBAL DEFAULT 11 sgesdd_
/usr/lib64/libopenblas64-r0.3.3.so: 7422: 0000000001a82b10 3110 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/libopenblas64-r0.3.3.so: 8860: 0000000001944960 19913 FUNC GLOBAL DEFAULT 12 sgesdd_
/usr/lib64/libopenblas64.so: 7422: 0000000001a82b10 3110 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/libopenblas64.so: 8860: 0000000001944960 19913 FUNC GLOBAL DEFAULT 12 sgesdd_
/usr/lib64/libopenblas64.so.0: 7422: 0000000001a82b10 3110 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/libopenblas64.so.0: 8860: 0000000001944960 19913 FUNC GLOBAL DEFAULT 12 sgesdd_
/usr/lib64/libopenblaso64-r0.3.3.so: 7619: 0000000001b035b0 3110 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/libopenblaso64-r0.3.3.so: 9095: 00000000019c5360 19913 FUNC GLOBAL DEFAULT 12 sgesdd_
/usr/lib64/libopenblaso64.so: 7619: 0000000001b035b0 3110 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/libopenblaso64.so: 9095: 00000000019c5360 19913 FUNC GLOBAL DEFAULT 12 sgesdd_
/usr/lib64/libopenblaso64.so.0: 7619: 0000000001b035b0 3110 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/libopenblaso64.so.0: 9095: 00000000019c5360 19913 FUNC GLOBAL DEFAULT 12 sgesdd_
/usr/lib64/libopenblaso-r0.3.3.so: 7617: 0000000001b0bb20 3160 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/libopenblaso-r0.3.3.so: 9093: 00000000019ccda0 19594 FUNC GLOBAL DEFAULT 12 sgesdd_
/usr/lib64/libopenblaso.so: 7617: 0000000001b0bb20 3160 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/libopenblaso.so: 9093: 00000000019ccda0 19594 FUNC GLOBAL DEFAULT 12 sgesdd_
/usr/lib64/libopenblaso.so.0: 7617: 0000000001b0bb20 3160 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/libopenblaso.so.0: 9093: 00000000019ccda0 19594 FUNC GLOBAL DEFAULT 12 sgesdd_
/usr/lib64/libopenblasp64-r0.3.3.so: 7626: 0000000001b03110 3110 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/libopenblasp64-r0.3.3.so: 9102: 00000000019c4f60 19913 FUNC GLOBAL DEFAULT 12 sgesdd_
/usr/lib64/libopenblasp64.so: 7626: 0000000001b03110 3110 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/libopenblasp64.so: 9102: 00000000019c4f60 19913 FUNC GLOBAL DEFAULT 12 sgesdd_
/usr/lib64/libopenblasp64.so.0: 7626: 0000000001b03110 3110 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/libopenblasp64.so.0: 9102: 00000000019c4f60 19913 FUNC GLOBAL DEFAULT 12 sgesdd_
/usr/lib64/libopenblasp-r0.3.3.so: 7624: 0000000001b0b680 3160 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/libopenblasp-r0.3.3.so: 9100: 00000000019cc9a0 19594 FUNC GLOBAL DEFAULT 12 sgesdd_
/usr/lib64/libopenblasp.so: 7624: 0000000001b0b680 3160 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/libopenblasp.so: 9100: 00000000019cc9a0 19594 FUNC GLOBAL DEFAULT 12 sgesdd_
/usr/lib64/libopenblasp.so.0: 7624: 0000000001b0b680 3160 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/libopenblasp.so.0: 9100: 00000000019cc9a0 19594 FUNC GLOBAL DEFAULT 12 sgesdd_
/usr/lib64/libopenblas-r0.3.3.so: 7420: 0000000001a83080 3160 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/libopenblas-r0.3.3.so: 8858: 00000000019443a0 19594 FUNC GLOBAL DEFAULT 12 sgesdd_
/usr/lib64/libopenblas.so: 7420: 0000000001a83080 3160 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/libopenblas.so: 8858: 00000000019443a0 19594 FUNC GLOBAL DEFAULT 12 sgesdd_
/usr/lib64/libopenblas.so.0: 7420: 0000000001a83080 3160 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/libopenblas.so.0: 8858: 00000000019443a0 19594 FUNC GLOBAL DEFAULT 12 sgesdd_
/usr/lib64/R/lib/libRblas.so: 7420: 0000000001a83080 3160 FUNC GLOBAL DEFAULT 12 dgebal_
/usr/lib64/R/lib/libRblas.so: 8858: 00000000019443a0 19594 FUNC GLOBAL DEFAULT 12 sgesdd_
so depending on your build then (maybe) you need to add -l<something>
or just pass one of the .so's above the link line at build time. For testing just hack it into the right place in your makefile or set LD_FLAGS='/lib/libfoo.so' before building to see if it works. If it does, then maybe something more graceful can be implemented that works across platforms.
I'm guessing you want one of these:
I've collected a bunch of LAPACKE/CLAPACK library names by distribution with comments and names in the mathlibs
structure here if that helps find the right library:
Thank you for this research! Are you able to repro @eserte's results of failing to find the function, then change that to success? I'd like to get the most idiomatic way to fix this in our Makefile.PL (and in a future Alien::LAPACK), since obviously setting LD_FLAGS and hacking makefiles is not a viable option.
ok so this fixes it for me at build time:
LD_PRELOAD='/usr/lib64/atlas/libtatlas.so.3' make test
@eserte, do you get these errors at perl Makefile.PL
?
Package 'lapack', required by 'virtual:world', not found
Package 'blas', required by 'virtual:world', not found
In Makefile.PL:10
it tries pkg-config
on the libs in @pkgs = qw(lapack blas)
but if those packages do not exist then it falls back to a default of '-L/usr/lib/atlas -llapack -lblas -latlas'
This fallback is probably the problem since it is unlikely to be valid when pkg-config
fails. I sort of think that Makefile.PL should just fail if pkg-config can't find the libraries. Also it might be best if @pkgs
was evaluated in order. Collect any errors and fail to finish Makefile.PL with the error report.
I'm still working on a final solution that builds here, so more to come...
Ok, so here are several solutions that build for me on Oracle Linux 8 (RHEL 8, etc). Here is what I found:
@pkgs
list in Makefile.PL don't do anything because all the .so's that need linked are in Real/ Trans/ and Complex/. Maybe it does something in win32? or maybe something needs to be done so the root Makefile.PL passes its library choices to the subdir Makefile.PL's?{Real,Trans,Complex}/Makefile.PL
files do not define libraries in %ldloadlibs
, but they need to. I tried each of these and they work for me but unfortunately they are distro specific:
So this is where my MakeMaker knowledge breaks down. Really what you want is to search for libraries in performance order and only choose one.
I'm mocking up a PR for comment.
make test
t/1.t ....... ok
t/cgtsv.t ... ok
t/gtsv.t .... ok
t/legacy.t .. ok
All tests successful.
Files=4, Tests=410, 3 wallclock secs ( 0.07 usr 0.01 sys + 2.59 cusr 0.13 csys = 2.80 CPU)
Result: PASS
We're discussing this on IRC, but I wanted to correct a misapprehension here for future generations.
The
{Real,Trans,Complex}/Makefile.PL
files do not define libraries in%ldloadlibs
, but they need to.
They do not need to, as that is not how subdirs' Makefile.PL
s work. They are each called in turn by EUMM as a result of the top-level M.PL's WriteMakefile()
, in the same process, so the various global variables (and defined functions) are available there.
Thanks for the correction. Commit message and branch have been updated: https://github.com/KJ7LNW/pdl-linearalgebra/commit/6f0742dfee57da7c20cb47a7e8655197f380eba5
I've merged @KJ7LNW's excellent PR (#17) and released the result as 0.34. Let's hope this fixes it.
Let's hope this fixes it.
The Debian package build for 0.34 failed due to test failures:
PERL_DL_NONLAZY=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(1, 'blib/lib', 'blib/arch')" t/*.t
Can't load '/build/libpdl-linearalgebra-perl-0.34/blib/arch/auto/PDL/LinearAlgebra/Real/Real.so' for module PDL::LinearAlgebra::Real: /build/libpdl-linearalgebra-perl-0.34/blib/arch/auto/PDL/LinearAlgebra/Real/Real.so: undefined symbol: sgesdd_ at /usr/lib/x86_64-linux-gnu/perl-base/DynaLoader.pm line 187.
at /build/libpdl-linearalgebra-perl-0.34/blib/lib/PDL/LinearAlgebra.pm line 9.
Compilation failed in require at /build/libpdl-linearalgebra-perl-0.34/blib/lib/PDL/LinearAlgebra.pm line 9.
BEGIN failed--compilation aborted at /build/libpdl-linearalgebra-perl-0.34/blib/lib/PDL/LinearAlgebra.pm line 9.
Compilation failed in require at t/1.t line 5.
BEGIN failed--compilation aborted at t/1.t line 5.
t/1.t .......
Dubious, test returned 2 (wstat 512, 0x200)
No subtests run
Can't load '/build/libpdl-linearalgebra-perl-0.34/blib/arch/auto/PDL/LinearAlgebra/Real/Real.so' for module PDL::LinearAlgebra::Real: /build/libpdl-linearalgebra-perl-0.34/blib/arch/auto/PDL/LinearAlgebra/Real/Real.so: undefined symbol: sgesdd_ at /usr/lib/x86_64-linux-gnu/perl-base/DynaLoader.pm line 187.
at complex.pd line 81.
Compilation failed in require at complex.pd line 81.
BEGIN failed--compilation aborted at complex.pd line 81.
Compilation failed in require at t/cgtsv.t line 8.
BEGIN failed--compilation aborted at t/cgtsv.t line 8.
t/cgtsv.t ...
Dubious, test returned 2 (wstat 512, 0x200)
No subtests run
Can't load '/build/libpdl-linearalgebra-perl-0.34/blib/arch/auto/PDL/LinearAlgebra/Real/Real.so' for module PDL::LinearAlgebra::Real: /build/libpdl-linearalgebra-perl-0.34/blib/arch/auto/PDL/LinearAlgebra/Real/Real.so: undefined symbol: sgesdd_ at /usr/lib/x86_64-linux-gnu/perl-base/DynaLoader.pm line 187.
at t/gtsv.t line 7.
Compilation failed in require at t/gtsv.t line 7.
BEGIN failed--compilation aborted at t/gtsv.t line 7.
t/gtsv.t ....
Dubious, test returned 2 (wstat 512, 0x200)
No subtests run
Can't load '/build/libpdl-linearalgebra-perl-0.34/blib/arch/auto/PDL/LinearAlgebra/Real/Real.so' for module PDL::LinearAlgebra::Real: /build/libpdl-linearalgebra-perl-0.34/blib/arch/auto/PDL/LinearAlgebra/Real/Real.so: undefined symbol: sgesdd_ at /usr/lib/x86_64-linux-gnu/perl-base/DynaLoader.pm line 187.
at /build/libpdl-linearalgebra-perl-0.34/blib/lib/PDL/LinearAlgebra.pm line 9.
Compilation failed in require at /build/libpdl-linearalgebra-perl-0.34/blib/lib/PDL/LinearAlgebra.pm line 9.
BEGIN failed--compilation aborted at /build/libpdl-linearalgebra-perl-0.34/blib/lib/PDL/LinearAlgebra.pm line 9.
Compilation failed in require at t/legacy.t line 6.
BEGIN failed--compilation aborted at t/legacy.t line 6.
t/legacy.t ..
Dubious, test returned 2 (wstat 512, 0x200)
No subtests run
Test Summary Report
-------------------
t/1.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: No plan found in TAP output
t/cgtsv.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: No plan found in TAP output
t/gtsv.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: No plan found in TAP output
t/legacy.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: No plan found in TAP output
Files=4, Tests=0, 2 wallclock secs ( 0.02 usr 0.02 sys + 2.18 cusr 0.06 csys = 2.28 CPU)
Result: FAIL
Failed 4/4 test programs. 0/0 subtests failed.
make[1]: *** [Makefile:934: test_dynamic] Error 2
make[1]: Leaving directory '/build/libpdl-linearalgebra-perl-0.34'
The 0.34 build used -llapack -lblas -latlas
which worked, 0.34 uses -llapack_atlas
which doesn't.
@sebastic Thanks for the report!
@KJ7LNW Looks like find_libs
needs to:
check_lib
to call sgesdd
(with appropriate use of the FORTRAN
macro)-lblas
Replacing libatlas-base-dev with libblas-dev & liblapack-dev resolves the issue for the Debian package.
Replacing both with libopenblas-dev or libopenblas-serial-dev also works.
Only the atlas implementation isn't supported by find_libs()
.
69fcefd2d7b74a765bb4f3aca4450711550459b9 seems like a reasonable fix. I've been working on a smarter find_libs() today but not yet ready for a PR. More when I am given time to look at it.
@KJ7LNW sounds good, but let's see how this current shape "plays" on CPAN testers. We may not need any further action.
Ok, @eserte 's systems are still failing on 0.35, e.g. https://www.cpantesters.org/cpan/report/da4cbd12-060a-11ed-a158-8ddd2cbfd570 (a Debian system) - I think find_libs
needs to do the full check_lib
for e.g. sgesdd
, ideally reusing the function
code written for the check_lib_or_exit
call, and it's probably worth permuting with a -lblas
too as mentioned before.
I also think we'll need the config system to write its LAPACK findings to a file that can be dumped by a standalone test so we can see what the system being tried is if a failure happens (and also when it succeeds).
@mohawk2 yep thats what I'm working on.
See pull request: https://github.com/PDLPorters/pdl-linearalgebra/pull/19
@mohawk2 ,
Looks like CI is failing on String::ShellQuote
but that was added to the CONFIGURE_REQUIRES
list. Can String::ShellQuote
be added to CI?
Looks like CI is failing on
String::ShellQuote
but that was added to theCONFIGURE_REQUIRES
list. CanString::ShellQuote
be added to CI?
Yes it can, and you can add it. See https://github.com/PDLPorters/pdl-linearalgebra/blob/61c04d4f9147d18f4ddb7b3760c817c63efa2db3/.github/workflows/ci.yml#L53
For other people ending up here with the same symptoms: you might not have this problem. It might be sufficient, after installing the relevant BLAS/LAPACK/ATLAS packages, to reinstall PDL::LinearAlgebra. Not to diminish the great work and investigation that's been going on so far, but to prevent other people from retracing my false steps!
@kqr Thank you for the feedback! A real problem is that PDL::LinearAlgebra is supposed to detect that situation, and is currently failing to do so.
I'm having similar problems. I'm on OpenSUSE Leap 15.5, both lapack-devel
and blas-devel
are installed. However, running perl Makefile.PL
outputs
Loaded ExtUtils::F77 version 1.26
Package lapack was not found in the pkg-config search path.
Perhaps you should add the directory containing `lapack.pc'
to the PKG_CONFIG_PATH environment variable
No package 'lapack' found
Package blas was not found in the pkg-config search path.
Perhaps you should add the directory containing `blas.pc'
to the PKG_CONFIG_PATH environment variable
No package 'blas' found
Checking if your kit is complete...
Looks good
Generating a Unix-style Makefile
Writing Makefile for PDL::LinearAlgebra
Writing MYMETA.yml and MYMETA.json
There are no lapack.pc
and blas.pc
files anywhere on the file system.
But the exit code is 0. When I proceed with make
, it runs without problems, but make test
fails terribly:
t/1.t ....... Can't load '/home/choroba/.cpan/build/PDL-LinearAlgebra-0.37-0/blib/arch/auto/PDL/LinearAlgebra/Real/Real.so' for module PDL::LinearAlgebra::Real: /home/choroba/.cpan/build/PDL-LinearAlgebra-0.37-0/blib/arch/auto/PDL/LinearAlgebra/Real/Real.so: undefined symbol: dgebal_ at /usr/lib/perl5/5.26.1/x86_64-linux-thread-multi/DynaLoader.pm line 193.
at /home/choroba/.cpan/build/PDL-LinearAlgebra-0.37-0/blib/lib/PDL/LinearAlgebra.pm line 9.
Compilation failed in require at /home/choroba/.cpan/build/PDL-LinearAlgebra-0.37-0/blib/lib/PDL/LinearAlgebra.pm line 9.
BEGIN failed--compilation aborted at /home/choroba/.cpan/build/PDL-LinearAlgebra-0.37-0/blib/lib/PDL/LinearAlgebra.pm line 9.
Compilation failed in require at t/1.t line 5.
BEGIN failed--compilation aborted at t/1.t line 5.
t/1.t ....... Dubious, test returned 2 (wstat 512, 0x200)
Interestingly, setting LD_PRELOAD=/usr/lib64/lapack/liblapack.so.3
solves the issue. As a temporary workaround, I added the export
line to my .bashrc
.
Interestingly, setting LD_PRELOAD=/usr/lib64/lapack/liblapack.so.3 solves the issue. As a temporary workaround, I added the export line to my .bashrc.
FYI: That will load liblapack.so.3
into all programs that you launch...
Does LD_LIBRARY_PATH=/usr/lib64/lapack/
work instead? If it does, then you might be able to specify that at build time, and then configure it to be permanent as follows:
echo /usr/lib64/lapack/ > /etc/ld.so.conf.d/atlas-x86_64.conf
ldconfig -a
Does LD_LIBRARY_PATH=/usr/lib64/lapack/ work instead?
Unfortunately, it doesn't.
https://github.com/PDLPorters/pdl-linearalgebra/issues/5 does not seem to be fixed in later PDL-LinearAlgebra versions (seen the error in 0.32):
or
Still Fedora is the only system amongst my smokers which has pass reports.