search

OpenMathLib / OpenBLAS

OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version.
http://www.openblas.net
BSD 3-Clause "New" or "Revised" License
6.14k stars 1.46k forks source link

A few openblas_utest_ext tests failed for cmake builds of OpenBLAS #4743

Closed azuresky01 closed 4 weeks ago

azuresky01 commented 1 month ago

Hello,

I tested current develop code of OpenBLAS on different platforms with different architectures [Loongson 3A6000 (loongarch64), AMD R9 7940HS (x86_64) ] and found the same 8 tests in openblas_utest_ext failed for cmake builds.

tested Operating system: Arch Linux, AOSC OS,...

Building commands: 

CC=gcc FC=gfortran cmake -B build
cmake --build build
cd build/utest
./openblas_utest_ext

Testing results on AMD R9 7940HS:

...
TEST 976/1526 zaxpby:c_api_check_n_zero [OK]
TEST 977/1526 caxpyc:conj_strides_one [FAIL]
  ERR: /home/test/OpenBLAS_test/git/test/utest/test_extensions/test_caxpyc.c:102  expected 0.000e+00, got 6.557e+01 (diff -6.557e+01, tol 1.000e-04)
TEST 978/1526 caxpyc:conj_incx_one [FAIL]
  ERR: /home/test/OpenBLAS_test/git/test/utest/test_extensions/test_caxpyc.c:120  expected 0.000e+00, got 6.580e+01 (diff -6.580e+01, tol 1.000e-04)
TEST 979/1526 caxpyc:conj_incy_one [FAIL]
  ERR: /home/test/OpenBLAS_test/git/test/utest/test_extensions/test_caxpyc.c:138  expected 0.000e+00, got 6.183e+01 (diff -6.183e+01, tol 1.000e-04)
TEST 980/1526 caxpyc:conj_strides_two [FAIL]
  ERR: /home/test/OpenBLAS_test/git/test/utest/test_extensions/test_caxpyc.c:156  expected 0.000e+00, got 6.176e+01 (diff -6.176e+01, tol 1.000e-04)
TEST 981/1526 zaxpyc:conj_strides_one [FAIL]
  ERR: /home/test/OpenBLAS_test/git/test/utest/test_extensions/test_zaxpyc.c:103  expected 0.000e+00, got 6.125e+01 (diff -6.125e+01, tol 1.000e-13)
TEST 982/1526 zaxpyc:conj_incx_one [FAIL]
  ERR: /home/test/OpenBLAS_test/git/test/utest/test_extensions/test_zaxpyc.c:121  expected 0.000e+00, got 6.253e+01 (diff -6.253e+01, tol 1.000e-13)
TEST 983/1526 zaxpyc:conj_incy_one [FAIL]
  ERR: /home/test/OpenBLAS_test/git/test/utest/test_extensions/test_zaxpyc.c:139  expected 0.000e+00, got 5.784e+01 (diff -5.784e+01, tol 1.000e-13)
TEST 984/1526 zaxpyc:conj_strides_two [FAIL]
  ERR: /home/test/OpenBLAS_test/git/test/utest/test_extensions/test_zaxpyc.c:157  expected 0.000e+00, got 6.173e+01 (diff -6.173e+01, tol 1.000e-13)
TEST 985/1526 cgemv:colmajor_trans_col_100_row_100_inc_x_1_y_1 [OK]
...
TEST 1526/1526 zsbmv:xerbla_lda_invalid [OK]
RESULTS: 1526 tests (1518 ok, 8 failed, 0 skipped) ran in 179 ms

on Loongson 3A6000:

...
TEST 977/1526 caxpyc:conj_strides_one [FAIL]
  ERR: /home/test/OpenBLAS_test/git/test2/utest/test_extensions/test_caxpyc.c:102  expected 0.000e+00, got 6.557e+01 (diff -6.557e+01, tol 1.000e-04)
TEST 978/1526 caxpyc:conj_incx_one [FAIL]
  ERR: /home/test/OpenBLAS_test/git/test2/utest/test_extensions/test_caxpyc.c:120  expected 0.000e+00, got 6.580e+01 (diff -6.580e+01, tol 1.000e-04)
TEST 979/1526 caxpyc:conj_incy_one [FAIL]
  ERR: /home/test/OpenBLAS_test/git/test2/utest/test_extensions/test_caxpyc.c:138  expected 0.000e+00, got 6.183e+01 (diff -6.183e+01, tol 1.000e-04)
TEST 980/1526 caxpyc:conj_strides_two [FAIL]
  ERR: /home/test/OpenBLAS_test/git/test2/utest/test_extensions/test_caxpyc.c:156  expected 0.000e+00, got 6.176e+01 (diff -6.176e+01, tol 1.000e-04)
TEST 981/1526 zaxpyc:conj_strides_one [FAIL]
  ERR: /home/test/OpenBLAS_test/git/test2/utest/test_extensions/test_zaxpyc.c:103  expected 0.000e+00, got 6.125e+01 (diff -6.125e+01, tol 1.000e-13)
TEST 982/1526 zaxpyc:conj_incx_one [FAIL]
  ERR: /home/test/OpenBLAS_test/git/test2/utest/test_extensions/test_zaxpyc.c:121  expected 0.000e+00, got 6.253e+01 (diff -6.253e+01, tol 1.000e-13)
TEST 983/1526 zaxpyc:conj_incy_one [FAIL]
  ERR: /home/test/OpenBLAS_test/git/test2/utest/test_extensions/test_zaxpyc.c:139  expected 0.000e+00, got 5.784e+01 (diff -5.784e+01, tol 1.000e-13)
TEST 984/1526 zaxpyc:conj_strides_two [FAIL]
  ERR: /home/test/OpenBLAS_test/git/test2/utest/test_extensions/test_zaxpyc.c:157  expected 0.000e+00, got 6.173e+01 (diff -6.173e+01, tol 1.000e-13)
...

If building with CC=gcc FC=gfortran make, all tests in openblas_utest_ext are OK.

martin-frbg commented 1 month ago

I don't recall seeing this, will investigate. Maybe it is the difference in optimization options (-O2 for make vs either nothing or -O3 for cmake, depending on whether you set CMAKE_BUILD_TYPE). Which gcc version did you use ?

azuresky01 commented 1 month ago

I don't recall seeing this, will investigate. Maybe it is the difference in optimization options (-O2 for make vs either nothing or -O3 for cmake, depending on whether you set CMAKE_BUILD_TYPE). Which gcc version did you use ?

It seems that gcc version does not matter. For AMD R9 7940HS gcc version is 14.1.1 (Arch Linux), for Loongson 3A6000 gcc version is 13.2.0 (AOSC OS). And I just tested on Debian 12 (gcc version 12.2.0) for AMD R9 7940HS, the result is the same. I did not set CMAKE_BUILD_TYPE.

azuresky01 commented 1 month ago

On Debian 12 I just tried different CMAKE_BUILD_TYPE with Debug, RelWithDebInfo or MinSizeRel. The 8 tests in openblas_utest_ext still failed.

martin-frbg commented 1 month ago

Reproduced now but have not identified any differences in build options for the relevant files yet. Curiously the executable sizes for openblas_utest_ext differ considerably between the two builds, so my current assumption is that the problem mainly/only affects the test itself

martin-frbg commented 4 weeks ago

Well, it had to be a dumb mistake I made in #4470 (affecting "only" 0.3.27)