search

flatironinstitute / cppdlr

Discrete Lehmann representation of imaginary time Green's functions
https://flatironinstitute.github.io/cppdlr/
Apache License 2.0
8 stars 4 forks source link

imfreq_ops and imtime_ops tests fail with Intel MKL #6

Closed hmenke closed 10 months ago

hmenke commented 10 months ago

Prerequisites

Description

The tests for imfreq_ops and imtime_ops fail the tolerance when run with Intel MKL. (Possibly related to #1?)

imfreq_ops ``` Running main() from /source/triqs.build/deps/GTest_src/googletest/src/gtest_main.cc [==========] Running 5 tests from 2 test suites. [----------] Global test environment set-up. [----------] 4 tests from imfreq_ops [ RUN ] imfreq_ops.interp_matrix eps = 1.000000e-10, Lambda = 1.000000e+03 Imag freq: l^2 err = 1.831801e-10, L^inf err = 8.042914e-09 /source/triqs.build/deps/cppdlr_src/test/c++/imfreq_ops.cpp:182: Failure Expected: (errlinf) < (100 * eps), actual: 1.0536994972037939e-08 vs 1e-08 /source/triqs.build/deps/cppdlr_src/test/c++/imfreq_ops.cpp:183: Failure Expected: (errl2) < (2 * eps), actual: 2.6854942394814565e-10 vs 2e-10 Imag time: L^2 err = 2.685494e-10, L^inf err = 1.053699e-08 [ FAILED ] imfreq_ops.interp_matrix (406 ms) [ RUN ] imfreq_ops.interp_scalar eps = 1.000000e-10, Lambda = 1.000000e+03 Imag freq: l^2 err = 7.170216e-11, L^inf err = 4.316508e-09 [ OK ] imfreq_ops.interp_scalar (251 ms) [ RUN ] imfreq_ops.interp_matrix_sym_fer eps = 1.000000e-10, Lambda = 1.000000e+03 Imag freq: l^2 err = 5.613459e-12, L^inf err = 5.393007e-10 Imag time: L^2 err = 6.595356e-12, L^inf err = 1.887863e-10 [ OK ] imfreq_ops.interp_matrix_sym_fer (405 ms) [ RUN ] imfreq_ops.interp_matrix_sym_bos eps = 1.000000e-10, Lambda = 1.000000e+03 Imag freq: l^2 err = 1.091396e-11, L^inf err = 1.859275e-09 Imag time: L^2 err = 1.405661e-11, L^inf err = 5.773454e-10 [ OK ] imfreq_ops.interp_matrix_sym_bos (521 ms) [----------] 4 tests from imfreq_ops (1602 ms total) [----------] 1 test from dlr_imfreq [ RUN ] dlr_imfreq.h5_rw [ OK ] dlr_imfreq.h5_rw (166 ms) [----------] 1 test from dlr_imfreq (166 ms total) [----------] Global test environment tear-down [==========] 5 tests from 2 test suites ran. (1769 ms total) [ PASSED ] 4 tests. [ FAILED ] 1 test, listed below: [ FAILED ] imfreq_ops.interp_matrix 1 FAILED TEST ```
imtime_ops ``` Running main() from /source/triqs.build/deps/GTest_src/googletest/src/gtest_main.cc [==========] Running 14 tests from 2 test suites. [----------] Global test environment set-up. [----------] 13 tests from imtime_ops [ RUN ] imtime_ops.interp_matrix eps = 1.000000e-10, Lambda = 1.000000e+03 /source/triqs.build/deps/cppdlr_src/test/c++/imtime_ops.cpp:165: Failure Expected: (errl2) < (eps), actual: 1.6029439334350401e-10 vs 1e-10 Imag time: L^2 err = 1.602944e-10, L^inf err = 8.505687e-10 /source/triqs.build/deps/cppdlr_src/test/c++/imtime_ops.cpp:183: Failure Expected: (errl2) < (eps), actual: 1.6030237139053056e-10 vs 1e-10 Imag freq: l^2 err = 1.603024e-10, L^inf err = 9.658213e-09 [ FAILED ] imtime_ops.interp_matrix (471 ms) [ RUN ] imtime_ops.interp_matrix_complex eps = 1.000000e-10, Lambda = 1.000000e+03 /source/triqs.build/deps/cppdlr_src/test/c++/imtime_ops.cpp:238: Failure Expected: (errl2) < (eps), actual: 1.6029439697593181e-10 vs 1e-10 Imag time: L^2 err = 1.602944e-10, L^inf err = 8.505687e-10 [ FAILED ] imtime_ops.interp_matrix_complex (183 ms) [ RUN ] imtime_ops.interp_scalar eps = 1.000000e-10, Lambda = 1.000000e+03 Imag time: L^2 err = 7.208404e-11, L^inf err = 5.949013e-10 [ OK ] imtime_ops.interp_scalar (136 ms) [ RUN ] imtime_ops.fit_matrix eps = 1.000000e-08, Lambda = 1.000000e+03, noise = 1.000000e-06 Imag time: L^2 err = 7.106571e-08, L^inf err = 8.355243e-07 [ OK ] imtime_ops.fit_matrix (177 ms) [ RUN ] imtime_ops.fit_matrix_cmplx eps = 1.000000e-08, Lambda = 1.000000e+03, noise = 1.000000e-06 Imag time: L^2 err = 1.390657e-06, L^inf err = 1.946862e-06 [ OK ] imtime_ops.fit_matrix_cmplx (215 ms) [ RUN ] imtime_ops.fit_scalar eps = 1.000000e-08, Lambda = 1.000000e+03, noise = 1.000000e-06 Imag time: L^2 err = 5.887686e-08, L^inf err = 9.043894e-07 [ OK ] imtime_ops.fit_scalar (124 ms) [ RUN ] imtime_ops.convolve_scalar_real eps = 1.000000e-12, Lambda = 1.000000e+03 /source/triqs.build/deps/cppdlr_src/test/c++/imtime_ops.cpp:532: Failure Expected: (errlinf) < (10 * eps), actual: 2.2399457288990732e-11 vs 9.9999999999999994e-12 /source/triqs.build/deps/cppdlr_src/test/c++/imtime_ops.cpp:533: Failure Expected: (errtlinf) < (10 * eps), actual: 2.2399554433505386e-11 vs 9.9999999999999994e-12 /source/triqs.build/deps/cppdlr_src/test/c++/imtime_ops.cpp:534: Failure Expected: (errl2) < (eps), actual: 3.1793264351165775e-12 vs 9.9999999999999998e-13 /source/triqs.build/deps/cppdlr_src/test/c++/imtime_ops.cpp:535: Failure Expected: (errtl2) < (eps), actual: 2.961859866378555e-12 vs 9.9999999999999998e-13 Ordinary convolution: L^inf err = 2.239946e-11, L^2 err = 3.179326e-12 Time-ordered convolution: L^inf err = 2.239955e-11, L^2 err = 2.961860e-12 [ FAILED ] imtime_ops.convolve_scalar_real (195 ms) [ RUN ] imtime_ops.convolve_scalar_cmplx eps = 1.000000e-12, Lambda = 1.000000e+03 /source/triqs.build/deps/cppdlr_src/test/c++/imtime_ops.cpp:618: Failure Expected: (errlinf) < (10 * eps), actual: 1.2444125591093425e-11 vs 9.9999999999999994e-12 /source/triqs.build/deps/cppdlr_src/test/c++/imtime_ops.cpp:619: Failure Expected: (errtlinf) < (10 * eps), actual: 1.2444229674501983e-11 vs 9.9999999999999994e-12 /source/triqs.build/deps/cppdlr_src/test/c++/imtime_ops.cpp:620: Failure Expected: (errl2) < (eps), actual: 1.7662826884969586e-12 vs 9.9999999999999998e-13 /source/triqs.build/deps/cppdlr_src/test/c++/imtime_ops.cpp:621: Failure Expected: (errtl2) < (eps), actual: 1.645477951212149e-12 vs 9.9999999999999998e-13 Ordinary convolution: L^inf err = 1.244413e-11, L^2 err = 1.766283e-12 Time-ordered convolution: L^inf err = 1.244423e-11, L^2 err = 1.645478e-12 [ FAILED ] imtime_ops.convolve_scalar_cmplx (182 ms) [ RUN ] imtime_ops.convolve_matrix_real eps = 1.000000e-12, Lambda = 1.000000e+03 /source/triqs.build/deps/cppdlr_src/test/c++/imtime_ops.cpp:708: Failure Expected: (errlinf) < (10 * eps), actual: 2.2399512800141963e-11 vs 9.9999999999999994e-12 /source/triqs.build/deps/cppdlr_src/test/c++/imtime_ops.cpp:709: Failure Expected: (errl2) < (2 * eps), actual: 6.3586095047429759e-12 vs 2e-12 /source/triqs.build/deps/cppdlr_src/test/c++/imtime_ops.cpp:710: Failure Expected: (errtlinf) < (10 * eps), actual: 2.2399443411202924e-11 vs 9.9999999999999994e-12 /source/triqs.build/deps/cppdlr_src/test/c++/imtime_ops.cpp:711: Failure Expected: (errtl2) < (2 * eps), actual: 5.9237008281555781e-12 vs 2e-12 Ordinary convolution: L^inf err = 2.239951e-11, L^2 err = 6.358610e-12 Time-ordered convolution: L^inf err = 2.239944e-11, L^2 err = 5.923701e-12 [ FAILED ] imtime_ops.convolve_matrix_real (282 ms) [ RUN ] imtime_ops.convolve_matrix_cmplx eps = 1.000000e-12, Lambda = 1.000000e+03 /source/triqs.build/deps/cppdlr_src/test/c++/imtime_ops.cpp:798: Failure Expected: (errlinf) < (10 * eps), actual: 1.2444163755009896e-11 vs 9.9999999999999994e-12 /source/triqs.build/deps/cppdlr_src/test/c++/imtime_ops.cpp:800: Failure Expected: (errtlinf) < (10 * eps), actual: 1.2444208857820271e-11 vs 9.9999999999999994e-12 Ordinary convolution: L^inf err = 1.244416e-11, L^2 err = 1.247914e-19 Time-ordered convolution: L^inf err = 1.244421e-11, L^2 err = 1.083039e-19 [ FAILED ] imtime_ops.convolve_matrix_cmplx (157 ms) [ RUN ] imtime_ops.refl_matrix eps = 1.000000e-10, Lambda = 1.000000e+01 Imag time: L^2 err = 2.817828e-11, L^inf err = 5.648537e-11 [ OK ] imtime_ops.refl_matrix (1 ms) [ RUN ] imtime_ops.interp_matrix_sym_fer eps = 1.000000e-10, Lambda = 1.000000e+03 Imag time: L^2 err = 1.434555e-11, L^inf err = 6.711230e-11 Imag freq: l^2 err = 1.434627e-11, L^inf err = 1.712101e-09 [ OK ] imtime_ops.interp_matrix_sym_fer (403 ms) [ RUN ] imtime_ops.interp_matrix_sym_bos eps = 1.000000e-10, Lambda = 1.000000e+03 Imag time: L^2 err = 1.399272e-11, L^inf err = 5.898525e-11 Imag freq: l^2 err = 1.399341e-11, L^inf err = 3.050253e-09 [ OK ] imtime_ops.interp_matrix_sym_bos (279 ms) [----------] 13 tests from imtime_ops (2811 ms total) [----------] 1 test from dlr_imtime [ RUN ] dlr_imtime.h5_rw [ OK ] dlr_imtime.h5_rw (50 ms) [----------] 1 test from dlr_imtime (50 ms total) [----------] Global test environment tear-down [==========] 14 tests from 2 test suites ran. (2861 ms total) [ PASSED ] 8 tests. [ FAILED ] 6 tests, listed below: [ FAILED ] imtime_ops.interp_matrix [ FAILED ] imtime_ops.interp_matrix_complex [ FAILED ] imtime_ops.convolve_scalar_real [ FAILED ] imtime_ops.convolve_scalar_cmplx [ FAILED ] imtime_ops.convolve_matrix_real [ FAILED ] imtime_ops.convolve_matrix_cmplx 6 FAILED TESTS ```

Steps to Reproduce

apt-get update
apt-get install build-essential cmake clang-15 git libc++-15-dev libc++abi-15-dev libhdf5-dev libmkl-full-dev libomp-15-dev libopenmpi-dev
git clone --branch main https://github.com/flatironinstitute/cppdlr
cd cppdlr/
export CPATH=/usr/include/mkl # Ubuntu's MKL CMake integration is broken
cmake -S . -B build/ -DCMAKE_CXX_COMPILER=clang++-15 -DCMAKE_CXX_FLAGS="-stdlib=libc++" -DBLA_VENDOR=Intel10_64_dyn -DMPIEXEC_PREFLAGS='--allow-run-as-root' -DCMAKE_INSTALL_PREFIX=$PWD/install
cmake --build build/
export CTEST_OUTPUT_ON_FAILURE=1
cmake --build build/ --target test

Expected behavior: Tests pass Actual behavior: Tests fail

Versions

https://github.com/flatironinstitute/cppdlr/commit/861df9396e01e650f84d63df3122a68280695cee

Ubuntu 22.04 jammy

Formatting

Please use markdown in your issue message. A useful summary of commands can be found here.

Additional Information

Any additional information, configuration or data that might be necessary to reproduce the issue.

hmenke commented 10 months ago

Easiest fix is probably to just bump the tolerances over the failing values.

Proposed patch ```diff diff --git a/test/c++/imfreq_ops.cpp b/test/c++/imfreq_ops.cpp index 0c81832..b55cfaf 100644 --- a/test/c++/imfreq_ops.cpp +++ b/test/c++/imfreq_ops.cpp @@ -179,8 +179,8 @@ TEST(imfreq_ops, interp_matrix) { } errl2 = sqrt(errl2 / ntst); - EXPECT_LT(errlinf, 100 * eps); - EXPECT_LT(errl2, 2 * eps); + EXPECT_LT(errlinf, 200 * eps); + EXPECT_LT(errl2, 3 * eps); std::cout << fmt::format("Imag time: L^2 err = {:e}, L^inf err = {:e}\n", errl2, errlinf); } diff --git a/test/c++/imtime_ops.cpp b/test/c++/imtime_ops.cpp index 19fc913..f5d2024 100644 --- a/test/c++/imtime_ops.cpp +++ b/test/c++/imtime_ops.cpp @@ -162,7 +162,7 @@ TEST(imtime_ops, interp_matrix) { errl2 = sqrt(errl2 / ntst); EXPECT_LT(errlinf, 10 * eps); - EXPECT_LT(errl2, eps); + EXPECT_LT(errl2, 2 * eps); std::cout << fmt::format("Imag time: L^2 err = {:e}, L^inf err = {:e}\n", errl2, errlinf); // Compute error in imaginary frequency @@ -180,7 +180,7 @@ TEST(imtime_ops, interp_matrix) { errl2 = sqrt(errl2) / beta; EXPECT_LT(errlinf, 100 * eps); - EXPECT_LT(errl2, eps); + EXPECT_LT(errl2, 2 * eps); std::cout << fmt::format("Imag freq: l^2 err = {:e}, L^inf err = {:e}\n", errl2, errlinf); } @@ -235,7 +235,7 @@ TEST(imtime_ops, interp_matrix_complex) { errl2 = sqrt(errl2 / ntst); EXPECT_LT(errlinf, 10 * eps); - EXPECT_LT(errl2, eps); + EXPECT_LT(errl2, 2 * eps); std::cout << fmt::format("Imag time: L^2 err = {:e}, L^inf err = {:e}\n", errl2, errlinf); } @@ -529,10 +529,10 @@ TEST(imtime_ops, convolve_scalar_real) { errl2 = sqrt(errl2 / ntst); errtl2 = sqrt(errtl2 / ntst); - EXPECT_LT(errlinf, 10 * eps); - EXPECT_LT(errtlinf, 10 * eps); - EXPECT_LT(errl2, eps); - EXPECT_LT(errtl2, eps); + EXPECT_LT(errlinf, 25 * eps); + EXPECT_LT(errtlinf, 25 * eps); + EXPECT_LT(errl2, 4 * eps); + EXPECT_LT(errtl2, 3 * eps); std::cout << fmt::format("Ordinary convolution: L^inf err = {:e}, L^2 err = {:e}\n", errlinf, errl2); std::cout << fmt::format("Time-ordered convolution: L^inf err = {:e}, L^2 err = {:e}\n", errtlinf, errtl2); } @@ -615,10 +615,10 @@ TEST(imtime_ops, convolve_scalar_cmplx) { errl2 = sqrt(errl2 / ntst); errtl2 = sqrt(errtl2 / ntst); - EXPECT_LT(errlinf, 10 * eps); - EXPECT_LT(errtlinf, 10 * eps); - EXPECT_LT(errl2, eps); - EXPECT_LT(errtl2, eps); + EXPECT_LT(errlinf, 13 * eps); + EXPECT_LT(errtlinf, 13 * eps); + EXPECT_LT(errl2, 2 * eps); + EXPECT_LT(errtl2, 2 * eps); std::cout << fmt::format("Ordinary convolution: L^inf err = {:e}, L^2 err = {:e}\n", errlinf, errl2); std::cout << fmt::format("Time-ordered convolution: L^inf err = {:e}, L^2 err = {:e}\n", errtlinf, errtl2); } @@ -705,10 +705,10 @@ TEST(imtime_ops, convolve_matrix_real) { errl2 = sqrt(errl2 / ntst); errtl2 = sqrt(errtl2 / ntst); - EXPECT_LT(errlinf, 10 * eps); - EXPECT_LT(errl2, 2 * eps); - EXPECT_LT(errtlinf, 10 * eps); - EXPECT_LT(errtl2, 2 * eps); + EXPECT_LT(errlinf, 23 * eps); + EXPECT_LT(errl2, 7 * eps); + EXPECT_LT(errtlinf, 23 * eps); + EXPECT_LT(errtl2, 6 * eps); std::cout << fmt::format("Ordinary convolution: L^inf err = {:e}, L^2 err = {:e}\n", errlinf, errl2); std::cout << fmt::format("Time-ordered convolution: L^inf err = {:e}, L^2 err = {:e}\n", errtlinf, errtl2); } @@ -795,9 +795,9 @@ TEST(imtime_ops, convolve_matrix_cmplx) { errtl2 += pow(frobenius_norm(gttru - gttst), 2); } - EXPECT_LT(errlinf, 10 * eps); + EXPECT_LT(errlinf, 13 * eps); EXPECT_LT(errl2, 2 * eps); - EXPECT_LT(errtlinf, 10 * eps); + EXPECT_LT(errtlinf, 13 * eps); EXPECT_LT(errtl2, 2 * eps); std::cout << fmt::format("Ordinary convolution: L^inf err = {:e}, L^2 err = {:e}\n", errlinf, errl2); std::cout << fmt::format("Time-ordered convolution: L^inf err = {:e}, L^2 err = {:e}\n", errtlinf, errtl2); ```
hmenke commented 10 months ago

Okay, increasing the tolerance seems to be the only option, because the failure depends on available CPU features. On a machine with AVX512 I was at first not able to reproduce, but then when setting MKL_CBWR to

export MKL_CBWR="BRANCH=COMPATIBLE,STRICT"

to turn off all vectorization extensions the failures showed up again.

jasonkaye commented 10 months ago

@Wentzell Could this have to do with the possible fast-math issues we were seeing?

We'll look into this soon. I'd like to understand a little better what's happening before bumping up test tolerances, as the user-specified tolerances should typically be satisfied or very nearly satisfied.

jasonkaye commented 10 months ago

Thanks for the pull request, this has been merged and should fix the issue.