Closed davydden closed 8 years ago
indeed,
$ git diff
diff --git a/var/spack/repos/builtin/packages/openblas/package.py b/var/spack/repos/builtin/packages/openblas/package.py
index 4ec829a..39c49db 100644
--- a/var/spack/repos/builtin/packages/openblas/package.py
+++ b/var/spack/repos/builtin/packages/openblas/package.py
@@ -19,8 +19,8 @@ class Openblas(Package):
def install(self, spec, prefix):
- make_defs = ['CC=%s' % spack_cc,
- 'FC=%s' % spack_fc]
+ make_defs = ['CC=cc',
+ 'FC=f77']
make_targets = ['libs', 'netlib']
@@ -67,4 +67,3 @@ class Openblas(Package):
if '+shared' in self.spec:
self.spec.blas_shared_lib = join_path(libdir, 'libopenblas.%s' % dso_suffix)
self.spec.lapack_shared_lib = self.spec.blas_shared_lib
-
fixes missing symbols in Openblas (i have not tried to build Trilinos though). Note, that the build time changes from 1m 37.33s
to 3m 42.35s
with this change.
@tgamblin do you recall why you changed those arguments in Openblas? My understanding is that CC and F77 point to compiler wrappers, whereas spack_cc
and spack_f77
are direct compilers. Why it makes that huge difference here -- i don't know.
@davydden FC=f77
seems a bit suspicious...
@davydden: sorry I haven't gotten to this yet.
Both actually point to compiler wrappers. Hardcoding CC=cc
and F77=f77
will always work in Spack; those are the generic compiler names. Originally, that is actually how we built most packages. spack_cc
, etc. are absolute paths to the named links to the compiler wrappers. Here is the directory structure:
lib/spack/env/
cc
c++ -> cc
f77 -> cc
f90 -> cc
gcc/
gcc -> ../cc
g++ .> ../cc
gfortran -> ../cc
cc
is just the name of the compiler wrapper; lib/spack/env
is in the build PATH
so invoking it will work. spack_cc
is the absolute path to lib/spack/env/gcc/gcc
, which also points to cc
. The name matters for libtool and other builds that try to deduce the compiler from the name (FWIW, I'm curious whether this is going to help us on Cray, where the builds see a more informative name than they would compiling directly with the Cray cc
/CC
/ftn
wrappers).
In the above example, the one thing I may have botched is changing f77
to spack_fc
. That should probably be spack_f77
. Do you mind trying that change out?
If that does not help, I think it means that the openblas
build is doing something wrong w.r.t. assumptions about the compiler name on Mac OS X (clang? gcc?). openblas
uses a pretty customized build so that is definitely possible... maybe cc
and f77
are better for this case, but I wonder if that will cause it to pick the wrong linker flags elsewhere (on Linux).
@alalazo @tgamblin : here are some clues variant 1 (not working):
==> make -j8 CC=cc FC=fc libs netlib shared
OpenBLAS: Detecting fortran compiler failed. Cannot compile LAPACK. Only compile BLAS.
make[1]: warning: -jN forced in submake: disabling jobserver mode.
variant 2 (working):
==> make -j8 libs netlib shared CC=cc FC=f77
make[1]: warning: -jN forced in submake: disabling jobserver mode.
variant 3 (not working):
==> make -j8 CC=/Users/davydden/spack/lib/spack/env/clang/clang FC=/Users/davydden/spack/lib/spack/env/f90 libs netlib shared
OpenBLAS: Detecting fortran compiler failed. Cannot compile LAPACK. Only compile BLAS.
make[1]: warning: -jN forced in submake: disabling jobserver mode.
so it boils down to understand how exactly openblas detects fortran compiler and why does it fail in variant 3 and variant 1 but works in variant 2.
p.s. spack_f77
does not work either:
*variant 4 (not working):
==> make -j8 CC=/Users/davydden/spack/lib/spack/env/clang/clang FC=/Users/davydden/spack/lib/spack/env/f90 libs netlib shared
OpenBLAS: Detecting fortran compiler failed. Cannot compile LAPACK. Only compile BLAS.
make[1]: warning: -jN forced in submake: disabling jobserver mode.
i thinks it is related to wrappers somehow... The same kind of hidden issues we had when CMake couldn't pick up working Fortran compiler for mixed C/Fortran projects: https://github.com/LLNL/spack/issues/634
@davydden: 4 looks the same as 3 to me. It says f90
, not f77
. Can you see if that one is correct? I suspect that one should work.
@tgamblin : you are right, i must have copy-pasted it from the wrong file
variant 4 (not working):
==> make -j8 CC=/Users/davydden/spack/lib/spack/env/clang/clang FC=/Users/davydden/spack/lib/spack/env/f77 libs netlib shared
OpenBLAS: Detecting fortran compiler failed. Cannot compile LAPACK. Only compile BLAS.
It looks like the build doesn't properly take the base name of the compiler to identify it. If I run the build with an absolute path to f77
it fails; if I run it with just f77
it succeeds. I tried this with this environment:
$ spack env openblas%clang bash
$ make CC=... FC=...
I think we want CC
to be called clang
, or the build won't get clang flags right. For f77
I can't do much, as compiler mixing isn't well supported right now so we use the generic name when compiling with clang.
Can you try making openblas/package.py
set the compilers to point to os.path.basename(spack_cc)
and os.path.basename(spack_f77)
? That will get you the proper names without reverting entirely to the generically named wrappers.
FYI: I switched to using absolute compiler paths when I discovered it can really affect build time. Using an abspath avoids searching PATH entries on machines where compilers are in slow shared filesystems, and it sped up builds by a surprising amount. Most builds handle abspaths fine, but openblas is a roll-your-own build. I am pretty sure I like the speed enough NOT to switch back to just using names, but if a lot of builds behave this way it might be good to make it easier to use just the compiler names.
@tgamblin : this
$ git diff
diff --git a/var/spack/repos/builtin/packages/openblas/package.py b/var/spack/repos/builtin/packages/openblas/package.py
index 4ec829a..7405de3 100644
--- a/var/spack/repos/builtin/packages/openblas/package.py
+++ b/var/spack/repos/builtin/packages/openblas/package.py
@@ -19,8 +19,8 @@ class Openblas(Package):
def install(self, spec, prefix):
- make_defs = ['CC=%s' % spack_cc,
- 'FC=%s' % spack_fc]
+ make_defs = ['CC=%s' % os.path.basename(spack_cc),
+ 'FC=%s' % os.path.basename(spack_f77)]
make_targets = ['libs', 'netlib']
@@ -67,4 +67,3 @@ class Openblas(Package):
if '+shared' in self.spec:
self.spec.blas_shared_lib = join_path(libdir, 'libopenblas.%s' % dso_suffix)
self.spec.lapack_shared_lib = self.spec.blas_shared_lib
-
worked
==> make -j8 CC=clang FC=f77 libs netlib shared
make[1]: warning: -jN forced in submake: disabling jobserver mode.
I can create a PR if that's what you meant.
yes please! And thanks for helping to diagnose this!
I gave it a quick try while working on PR and noticed that Trilinos (as of f5a77d39580d215f8d4948b2cbe6c8d47d4fd514) does not build on OSX due to missing
blas
/lapack
symbols, see below. I am quite certain that it is related to recent changes inopenblas
https://github.com/LLNL/spack/commit/66bb71534b3ab7eaf7b27743efc0ecf9763e4982 . Possibly usage of thisinstead of
Otherwise I do not see anything that could make a noticeable difference.
Carefull examination confirms the missing symbols in
openblas
: current version:previous version:
The issue could be related to compiler wrappers and maybe somehow part of the
openblas
was not actually build. I must say that Trilinos builds on Ubuntu 14.04 with the current version of openblas.This brings us back to the discussion of checking symbols. I think we should take from somewhere (even from the errors below for starters) the list of blas/lapack symbols and do sanity checks within each package that implements blas/lapack. Perhaps also make Travis build modified packages in each PR so that those type of issues appear sooner.
p.s. Actual linking error in Trilinos: