Open h-vetinari opened 3 years ago
conda-forge-linter
commented
3 years ago Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.
h-vetinari
commented
3 years ago @martin-frbg, I wanted to contribute to the debugging of https://github.com/xianyi/OpenBLAS/pull/3308 (now on the correct feedstock compared to https://github.com/conda-forge/blas-feedstock/pull/74), but it's perhaps the first time the lapack-test suite is being run here, and it seems to be picking up a separate problem.
Namely, even with https://github.com/xianyi/OpenBLAS/pull/3250 reverted (though the result is the same without reversion), I get a compilation during make lapack-test on OSX (interestingly, only for INTERFACE64=1):
$BUILD_PREFIX/bin/x86_64-apple-darwin13.4.0-gfortran -pthread -march=core2 -mtune=haswell -ftree-vectorize -fPIC -fstack-protector -O2 -pipe -isystem $PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/openblas-0.3.16 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -frecursive -O2 -m128bit-long-double -Wall -frecursive -fno-optimize-sibling-calls -m64 -fdefault-integer-8 -fopenmp -fPIC -march=core2 -mtune=haswell -ftree-vectorize -fPIC -fstack-protector -O2 -pipe -isystem $PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/openblas-0.3.16 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -frecursive -O2 -m128bit-long-double -Wall -frecursive -fno-optimize-sibling-calls -m64 -fdefault-integer-8 -fopenmp -fPIC -lpthread -lgfortran -lomp -o xeigtstc cchkee.o cbdt01.o cbdt02.o cbdt03.o cbdt05.o cchkbb.o cchkbd.o cchkbk.o cchkbl.o cchkec.o cchkgg.o cchkgk.o cchkgl.o cchkhb.o cchkhs.o cchkst.o cchkst2stg.o cchkhb2stg.o cckcsd.o cckglm.o cckgqr.o cckgsv.o ccklse.o ccsdts.o cdrges.o cdrgev.o cdrges3.o cdrgev3.o cdrgsx.o cdrgvx.o cdrvbd.o cdrves.o cdrvev.o cdrvsg.o cdrvsg2stg.o cdrvst.o cdrvst2stg.o cdrvsx.o cdrvvx.o cerrbd.o cerrec.o cerred.o cerrgg.o cerrhs.o cerrst.o cget02.o cget10.o cget22.o cget23.o cget24.o cget35.o cget36.o cget37.o cget38.o cget51.o cget52.o cget54.o cglmts.o cgqrts.o cgrqts.o cgsvts3.o chbt21.o chet21.o chet22.o chpt21.o chst01.o clarfy.o clarhs.o clatm4.o clctes.o clctsx.o clsets.o csbmv.o csgt01.o cslect.o cstt21.o cstt22.o cunt01.o cunt03.o slafts.o slahd2.o slasum.o slatb9.o sstech.o sstect.o ssvdch.o ssvdct.o ssxt1.o alahdg.o alasum.o alasvm.o alareq.o ilaenv.o xerbla.o xlaenv.o chkxer.o ../../../libopenblas64_p-r0.3.16.a
Undefined symbols for architecture x86_64:
"_omp_set_num_threads_8_", referenced from:
_MAIN__ in cchkee.o
ld: symbol(s) not found for architecture x86_64
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:134: xeigtstc] Error 1
make[1]: Leaving directory '$SRC_DIR/lapack-netlib/TESTING/EIG'
make: *** [Makefile:355: lapack-test] Error 2
martin-frbg
commented
3 years ago That is a different problem - it is lacking the information that it needs to link against OpenMP. Please try repeating the build options on the make lapack-test line, i.e. something like make USE_OPENMP=1 INTERFACE64=1 lapack-test
h-vetinari
commented
3 years ago That still kinda didn't work, but maybe I'm doing something wrong.
martin-frbg
commented
3 years ago Hm, strange. Could it be that this symbol is not present in the Apple/LLVM OpenMP library ? LAPACK tests actively setting the number of threads they want to run with is a new feature from 3.9.x, and I think the _8 version is the fortran wrapper for a long argument.
h-vetinari
commented
3 years ago I think the
_8version is the fortran wrapper for alongargument.
Not sure if I'm reading this correctly, but it seems LLVM openmp only has an int version: https://github.com/llvm/llvm-project/blob/llvmorg-12.0.1/openmp/runtime/src/include/omp.h.var#L57
martin-frbg
commented
3 years ago Then it is probably the combination of -fdefault-interface-8 for gfortran and LLVM openmp. I'll do a test build with Reference-LAPACK to make sure it happens there as well.
h-vetinari
commented
3 years ago @martin-frbg, I saw https://github.com/xianyi/OpenBLAS/pull/3311 & https://github.com/xianyi/OpenBLAS/pull/3312 - thanks! - which I could carry as patches here. ~Are you planning to pick up the changes for from https://github.com/Reference-LAPACK/lapack/pull/597 for openblas as well? Otherwise I cannot really add the lapack-tests to the CI here.~ Nevermind, saw https://github.com/xianyi/OpenBLAS/pull/3314, cool!
martin-frbg
commented
3 years ago Have not merged that yet as I am still trying to reproduce the problem - on Azure, a build with USE_OPENMP and INTERFACE64 compiles and links just fine with gfortran-10 and Apple clang 12 from xcode 12.4 but I have not established which flavor of libomp it is linking against ( need to find out what's the equivalent of ldd on OSX)
isuruf
commented
3 years ago ( need to find out what's the equivalent of ldd on OSX)
otool -L
h-vetinari
commented
3 years ago @isuruf
I haven't added the make lapack-test to the windows builds, because I'm not sure exactly how to talk to make through cmake. Would something like this work?
diff --git a/recipe/bld.bat b/recipe/bld.bat
index 823c08c..2707e54 100644
--- a/recipe/bld.bat
+++ b/recipe/bld.bat
@@ -16,3 +16,17 @@ cmake -G "NMake Makefiles JOM" ^
jom install -j%CPU_COUNT%
utest\openblas_utest.exe
+
+cmake -G "NMake Makefiles JOM" ^
+ -DCMAKE_C_COMPILER=clang-cl ^
+ -DCMAKE_Fortran_COMPILER=flang ^
+ -DCMAKE_BUILD_TYPE=Release ^
+ -DCMAKE_INSTALL_PREFIX=%LIBRARY_PREFIX% ^
+ -DDYNAMIC_ARCH=ON ^
+ -DBUILD_WITHOUT_LAPACK=no ^
+ -DNOFORTRAN=0 ^
+ -DNUM_THREADS=128 ^
+ -DBUILD_SHARED_LIBS=on ^
+ %SRC_DIR% lapack-test
+
+jom install -j%CPU_COUNT%
+
+lapack_test\openblas_lapack_test.exe(copied from here and just adding lapack-test; not sure whether this actually needs the jom install resp. how to know in advance what the .exe will be called)
martin-frbg
commented
3 years ago I think the lapack testsuite should get built automatically when using cmake (though Reference-LAPACK has an extra option BUILD_TESTING for that), the testsuite should run with the other tests when ctest is invoked
h-vetinari
commented
3 years ago Interestingly, the missing symbol problem does not go away with (the current state of) https://github.com/xianyi/OpenBLAS/pull/3314
isuruf
commented
3 years ago Try the following patch.
diff --git a/lapack-netlib/lapack-netlib/TESTING/EIG/cchkee.F b/lapack-netlib/TESTING/EIG/cchkee.F
index 1595a3d1f..7c6f58087 100644
--- a/lapack-netlib/lapack-netlib/TESTING/EIG/cchkee.F
+++ b/lapack-netlib/TESTING/EIG/cchkee.F
@@ -1073,7 +1073,7 @@
INTEGER I, I1, IC, INFO, ITMP, K, LENP, MAXTYP, NEWSD,
$ NK, NN, NPARMS, NRHS, NTYPES,
$ VERS_MAJOR, VERS_MINOR, VERS_PATCH
- INTEGER*4 N_THREADS
+ INTEGER*4 N_THREADS, ONE_THREAD
REAL EPS, S1, S2, THRESH, THRSHN
* ..
* .. Local Arrays ..
@@ -1870,7 +1870,8 @@
IF( TSTERR ) THEN
#if defined(_OPENMP)
N_THREADS = OMP_GET_MAX_THREADS()
- CALL OMP_SET_NUM_THREADS(1)
+ ONE_THREAD = 1
+ CALL OMP_SET_NUM_THREADS(ONE_THREAD)
#endif
CALL CERRST( 'CST', NOUT )
#if defined(_OPENMP)
@@ -2337,7 +2338,8 @@
IF( TSTERR ) THEN
#if defined(_OPENMP)
N_THREADS = OMP_GET_MAX_THREADS()
- CALL OMP_SET_NUM_THREADS(1)
+ ONE_THREAD = 1
+ CALL OMP_SET_NUM_THREADS(ONE_THREAD)
#endif
CALL CERRST( 'CHB', NOUT )
#if defined(_OPENMP)
diff --git a/lapack-netlib/lapack-netlib/TESTING/EIG/dchkee.F b/lapack-netlib/TESTING/EIG/dchkee.F
index 3c40b30b3..2b8e0b371 100644
--- a/lapack-netlib/lapack-netlib/TESTING/EIG/dchkee.F
+++ b/lapack-netlib/TESTING/EIG/dchkee.F
@@ -1079,7 +1079,7 @@
INTEGER I, I1, IC, INFO, ITMP, K, LENP, MAXTYP, NEWSD,
$ NK, NN, NPARMS, NRHS, NTYPES,
$ VERS_MAJOR, VERS_MINOR, VERS_PATCH
- INTEGER*4 N_THREADS
+ INTEGER*4 N_THREADS, ONE_THREAD
DOUBLE PRECISION EPS, S1, S2, THRESH, THRSHN
* ..
* .. Local Arrays ..
@@ -1875,7 +1875,8 @@
IF( TSTERR ) THEN
#if defined(_OPENMP)
N_THREADS = OMP_GET_MAX_THREADS()
- CALL OMP_SET_NUM_THREADS(1)
+ ONE_THREAD = 1
+ CALL OMP_SET_NUM_THREADS(ONE_THREAD)
#endif
CALL DERRST( 'DST', NOUT )
#if defined(_OPENMP)
diff --git a/lapack-netlib/lapack-netlib/TESTING/EIG/schkee.F b/lapack-netlib/TESTING/EIG/schkee.F
index 70c54727d..bf04b5e5b 100644
--- a/lapack-netlib/lapack-netlib/TESTING/EIG/schkee.F
+++ b/lapack-netlib/TESTING/EIG/schkee.F
@@ -1079,7 +1079,7 @@
INTEGER I, I1, IC, INFO, ITMP, K, LENP, MAXTYP, NEWSD,
$ NK, NN, NPARMS, NRHS, NTYPES,
$ VERS_MAJOR, VERS_MINOR, VERS_PATCH
- INTEGER*4 N_THREADS
+ INTEGER*4 N_THREADS, ONE_THREAD
REAL EPS, S1, S2, THRESH, THRSHN
* ..
* .. Local Arrays ..
@@ -1876,7 +1876,8 @@
IF( TSTERR ) THEN
#if defined(_OPENMP)
N_THREADS = OMP_GET_MAX_THREADS()
- CALL OMP_SET_NUM_THREADS(1)
+ ONE_THREAD = 1
+ CALL OMP_SET_NUM_THREADS(ONE_THREAD)
#endif
CALL SERRST( 'SST', NOUT )
#if defined(_OPENMP)
diff --git a/lapack-netlib/lapack-netlib/TESTING/EIG/zchkee.F b/lapack-netlib/TESTING/EIG/zchkee.F
index de36cba33..b14127553 100644
--- a/lapack-netlib/lapack-netlib/TESTING/EIG/zchkee.F
+++ b/lapack-netlib/TESTING/EIG/zchkee.F
@@ -1073,7 +1073,7 @@
INTEGER I, I1, IC, INFO, ITMP, K, LENP, MAXTYP, NEWSD,
$ NK, NN, NPARMS, NRHS, NTYPES,
$ VERS_MAJOR, VERS_MINOR, VERS_PATCH
- INTEGER*4 N_THREADS
+ INTEGER*4 N_THREADS, ONE_THREAD
DOUBLE PRECISION EPS, S1, S2, THRESH, THRSHN
* ..
* .. Local Arrays ..
@@ -1870,7 +1870,8 @@
IF( TSTERR ) THEN
#if defined(_OPENMP)
N_THREADS = OMP_GET_MAX_THREADS()
- CALL OMP_SET_NUM_THREADS(1)
+ ONE_THREAD = 1
+ CALL OMP_SET_NUM_THREADS(ONE_THREAD)
#endif
CALL ZERRST( 'ZST', NOUT )
#if defined(_OPENMP)
@@ -2335,7 +2336,8 @@
IF( TSTERR ) THEN
#if defined(_OPENMP)
N_THREADS = OMP_GET_MAX_THREADS()
- CALL OMP_SET_NUM_THREADS(1)
+ ONE_THREAD = 1
+ CALL OMP_SET_NUM_THREADS(ONE_THREAD)
#endif
CALL ZERRST( 'ZHB', NOUT )
#if defined(_OPENMP)I had looked at where this might be coming from, but hadn't thought that 1 might be INTEGER by default. Good catch!
h-vetinari
commented
3 years ago with xianyi/OpenBLAS#3311, xianyi/OpenBLAS#3312 & xianyi/OpenBLAS#3314 & Isuru's patch (will you update Reference-LAPACK/lapack#597?), this now runs through.
@martin-frbg, are you considering a bugfix release? Otherwise I don't mind carrying patches on the feedstock here for the time being.
martin-frbg
commented
3 years ago I'm not sure if it actually ran through successfully, thought I saw a number of weird "OMP Error 178: Function sem_post failed" in the testsuite output, probably whenever the set_num_threads was in use ? (BTW my test worked simply because it found and linked both libgomp and libomp - something I try to avoid, but seems inevitably implied by gfortran's -fopenmp)
I do not think I have a choice regarding a quick 0.3.17 (but probably without this LAPACK OpenMP thing) unless I wanted to watch everything from NumPy to GIMP blowing up :frowning_face:
h-vetinari
commented
3 years ago I'm not sure if it actually ran through successfully, thought I saw a number of weird "OMP Error 178: Function sem_post failed" in the testsuite output, probably whenever the set_num_threads was in use?
You're right, I had overlooked that. Actually, even with set -e, this ran through, so I'm guessing the errors are not being reacted to correctly on the makefile-side?
--> LAPACK TESTING SUMMARY <--
SUMMARY nb test run numerical error other error
================ =========== ================= ================
REAL 1174263 2 (0.000%) 0 (0.000%)
DOUBLE PRECISION 1227437 0 (0.000%) 0 (0.000%)
COMPLEX 694951 1 (0.000%) 0 (0.000%)
COMPLEX16 696320 2 (0.000%) 0 (0.000%)
--> ALL PRECISIONS 3792971 5 (0.000%) 0 (0.000%) I do not think I have a choice regarding a quick 0.3.17 (but probably without this LAPACK OpenMP thing) unless I wanted to watch everything from NumPy to GIMP blowing up ☹️
thanks for the quick handling of all this! 😊
h-vetinari
commented
3 years ago @isuruf, any idea how to fix the current "OMP Error 178: Function sem_post failed"? I'm a bit stumped here, but would like to help unblock this (and the new blas builds...)
h-vetinari
commented
2 years ago the osx thing should be gone with https://github.com/conda-forge/openmp-feedstock/pull/51, but that still leaves the windows failure. Should we merge this PR (after clean-up) just for linxu/osx?
h-vetinari
commented
2 years ago @martin-frbg, this runs into https://github.com/xianyi/OpenBLAS/issues/3406 (though there are comparatively few failures on osx, and only 1 on linux), but on windows, the situation looks a bit worse:
93% tests passed, 8 tests failed out of 115
Total Test time (real) = 667.65 sec
The following tests FAILED:
3 - sblas2 (Failed)
4 - sblas3 (Failed)
6 - dblas2 (Failed)
7 - dblas3 (Failed)
Errors while running CTest
9 - cblas2 (Failed)
10 - cblas3 (Failed)
12 - zblas2 (Failed)
13 - zblas3 (Failed)Any thoughts about this...?
Trying to debug https://github.com/conda-forge/blas-feedstock/pull/74