search

RobotLocomotion / drake

Model-based design and verification for robotics.
https://drake.mit.edu
Other
3.34k stars 1.27k forks source link

Unit test failure in solvers:csdp_solver_test #18635

Closed ajaygunalan closed 1 year ago

ajaygunalan commented 1 year ago

What happened?

I followed these instrcutions:

cd /path/to/drake
bazel build //...                 # Build the entire project.
bazel test //...                  # Build and test the entire project.

But, I got the following error:

[----------] Global test environment tear-down
[==========] 31 tests from 17 test suites ran. (123 ms total)
[  PASSED  ] 30 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] TestSOS.MotzkinPolynomial

 1 FAILED TEST
================================================================================
INFO: Elapsed time: 1.431s, Critical Path: 0.32s
INFO: 3 processes: 1 internal, 1 linux-sandbox, 1 local.
INFO: Build completed, 1 test FAILED, 3 total actions
//solvers:csdp_solver_test                                               FAILED in 0.2s
  /home/aj/.cache/bazel/_bazel_aj/84dff73cad498d702e4464d2c661c905/execroot/drake/bazel-out/k8-opt/testlogs/solvers/csdp_solver_test/test.log

Executed 2 out of 6054 tests: 6053 tests pass and 1 fails locally.
INFO: Build completed, 1 test FAILED, 3 total actions

Version

Cloned the latest drake from the GitHub repository.

What operating system are you using?

Ubuntu 20.04

What installation option are you using?

compiled from source code using Bazel

Relevant log output

Here is the link to my test.log file.

jwnimmer-tri commented 1 year ago

That test is passing for the whole team locally, as well as in our automated build farm under many different configurations. There must be something different between your system and our systems.

Here's my output of ldd bazel-bin/solvers/csdp_solver_test on Ubuntu 20.04. Could you check if your output is different in any meaningful way (i.e., other than the home directory path to my Drake checkout)?

$ ldd bazel-bin/solvers/csdp_solver_test
    linux-vdso.so.1
    libexternal_Spicosat_Slibpicosat.so => /home/jwnimmer/jwnimmer-tri/drake/bazel-bin/solvers/../_solib_k8/libexternal_Spicosat_Slibpicosat.so
    libexternal_Scds_Slibcds.so => /home/jwnimmer/jwnimmer-tri/drake/bazel-bin/solvers/../_solib_k8/libexternal_Scds_Slibcds.so
    libexternal_Sdreal_Sthird_Uparty_Scom_Ugithub_Uprogschj_Uthreadpool_Slibthread_Upool.so => /home/jwnimmer/jwnimmer-tri/drake/bazel-bin/solvers/../_solib_k8/libexternal_Sdreal_Sthird_Uparty_Scom_Ugithub_Uprogschj_Uthreadpool_Slibthread_Upool.so
    libexternal_Sdreal_Sthird_Uparty_Scom_Ugithub_Urobotlocomotion_Udrake_Slibdrake_Usymbolic.so => /home/jwnimmer/jwnimmer-tri/drake/bazel-bin/solvers/../_solib_k8/libexternal_Sdreal_Sthird_Uparty_Scom_Ugithub_Urobotlocomotion_Udrake_Slibdrake_Usymbolic.so
    libdrake_ibex.so => /home/jwnimmer/jwnimmer-tri/drake/bazel-bin/solvers/../_solib_k8/_U@ibex_S_S_Cibex___Uexternal_Sibex/libdrake_ibex.so
    libexternal_Sgflags_Slibgflags.so => /home/jwnimmer/jwnimmer-tri/drake/bazel-bin/solvers/../_solib_k8/libexternal_Sgflags_Slibgflags.so
    libClpSolver.so.1 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libClpSolver.so.1
    libClp.so.1 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libClp.so.1
    libCoinUtils.so.3 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libCoinUtils.so.3
    libbz2.so.1.0 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbz2.so.1.0
    libz.so.1 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libz.so.1
    liblapack.so.3 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/liblapack.so.3
    libblas.so.3 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libblas.so.3
    libm.so.6 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libm.so.6
    libipopt.so.1 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../../lib/libipopt.so.1
    libdmumps_seq-5.2.1.so => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libdmumps_seq-5.2.1.so
    libdl.so.2 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libdl.so.2
    libgfortran.so.5 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libgfortran.so.5
    libquadmath.so.0 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libquadmath.so.0
    librt.so.1 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/librt.so.1
    libamd.so.2 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libamd.so.2
    libsuitesparseconfig.so.5 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libsuitesparseconfig.so.5
    libspdlog.so.1 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libspdlog.so.1
    libpthread.so.0 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libpthread.so.0
    libstdc++.so.6 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libstdc++.so.6
    libgcc_s.so.1 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libgcc_s.so.1
    libc.so.6 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc.so.6
    /lib64/ld-linux-x86-64.so.2
    libmumps_common_seq-5.2.1.so => /lib/x86_64-linux-gnu/libmumps_common_seq-5.2.1.so
    libpord_seq-5.2.1.so => /lib/x86_64-linux-gnu/libpord_seq-5.2.1.so
    libesmumps-6.so => /lib/x86_64-linux-gnu/libesmumps-6.so
    libscotch-6.so => /lib/x86_64-linux-gnu/libscotch-6.so
    libscotcherr-6.so => /lib/x86_64-linux-gnu/libscotcherr-6.so
    liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5

Of additional note would also be the libblas and liblapack libraries:

$ file /etc/alternatives/libblas.so.3-x86_64-linux-gnu 
/etc/alternatives/libblas.so.3-x86_64-linux-gnu: symbolic link to /usr/lib/x86_64-linux-gnu/blas/libblas.so.3
$ file /etc/alternatives/liblapack.so.3-x86_64-linux-gnu 
/etc/alternatives/liblapack.so.3-x86_64-linux-gnu: symbolic link to /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3
ajaygunalan commented 1 year ago

Yeah, sure! I checked it, and it seems fine to me!

Here is my output:

aj@ajIIT:~/drake$ ldd bazel-bin/solvers/csdp_solver_test
    linux-vdso.so.1 (0x00007ffdd419e000)
    libexternal_Spicosat_Slibpicosat.so => /home/aj/drake/bazel-bin/solvers/../_solib_k8/libexternal_Spicosat_Slibpicosat.so (0x00007f307309f000)
    libexternal_Scds_Slibcds.so => /home/aj/drake/bazel-bin/solvers/../_solib_k8/libexternal_Scds_Slibcds.so (0x00007f3073093000)
    libexternal_Sdreal_Sthird_Uparty_Scom_Ugithub_Uprogschj_Uthreadpool_Slibthread_Upool.so => /home/aj/drake/bazel-bin/solvers/../_solib_k8/libexternal_Sdreal_Sthird_Uparty_Scom_Ugithub_Uprogschj_Uthreadpool_Slibthread_Upool.so (0x00007f3073090000)
    libexternal_Sdreal_Sthird_Uparty_Scom_Ugithub_Urobotlocomotion_Udrake_Slibdrake_Usymbolic.so => /home/aj/drake/bazel-bin/solvers/../_solib_k8/libexternal_Sdreal_Sthird_Uparty_Scom_Ugithub_Urobotlocomotion_Udrake_Slibdrake_Usymbolic.so (0x00007f307302d000)
    libdrake_ibex.so => /home/aj/drake/bazel-bin/solvers/../_solib_k8/_U@ibex_S_S_Cibex___Uexternal_Sibex/libdrake_ibex.so (0x00007f3072dcf000)
    libexternal_Sgflags_Slibgflags.so => /home/aj/drake/bazel-bin/solvers/../_solib_k8/libexternal_Sgflags_Slibgflags.so (0x00007f3072da6000)
    libClpSolver.so.1 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libClpSolver.so.1 (0x00007f3072d3f000)
    libClp.so.1 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libClp.so.1 (0x00007f3072b62000)
    libCoinUtils.so.3 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libCoinUtils.so.3 (0x00007f3072a15000)
    libbz2.so.1.0 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbz2.so.1.0 (0x00007f3072a02000)
    libz.so.1 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libz.so.1 (0x00007f30729e4000)
    liblapack.so.3 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/liblapack.so.3 (0x00007f307231c000)
    libblas.so.3 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libblas.so.3 (0x00007f3071f56000)
    libm.so.6 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libm.so.6 (0x00007f3071e07000)
    libipopt.so.1 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../../lib/libipopt.so.1 (0x00007f3071b9a000)
    libdmumps_seq-5.2.1.so => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libdmumps_seq-5.2.1.so (0x00007f307198b000)
    libdl.so.2 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libdl.so.2 (0x00007f3071983000)
    libgfortran.so.5 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libgfortran.so.5 (0x00007f30716bb000)
    libquadmath.so.0 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libquadmath.so.0 (0x00007f3071671000)
    librt.so.1 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/librt.so.1 (0x00007f3071667000)
    libamd.so.2 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libamd.so.2 (0x00007f307165c000)
    libsuitesparseconfig.so.5 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libsuitesparseconfig.so.5 (0x00007f3071655000)
    libspdlog.so.1 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libspdlog.so.1 (0x00007f30715ac000)
    libpthread.so.0 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libpthread.so.0 (0x00007f3071589000)
    libstdc++.so.6 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libstdc++.so.6 (0x00007f30713a7000)
    libgcc_s.so.1 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libgcc_s.so.1 (0x00007f307138c000)
    libc.so.6 => /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc.so.6 (0x00007f307119a000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f30732c8000)
    libmumps_common_seq-5.2.1.so => /lib/x86_64-linux-gnu/libmumps_common_seq-5.2.1.so (0x00007f3071105000)
    libpord_seq-5.2.1.so => /lib/x86_64-linux-gnu/libpord_seq-5.2.1.so (0x00007f30710ec000)
    libesmumps-6.so => /lib/x86_64-linux-gnu/libesmumps-6.so (0x00007f30710e6000)
    libscotch-6.so => /lib/x86_64-linux-gnu/libscotch-6.so (0x00007f3071051000)
    libscotcherr-6.so => /lib/x86_64-linux-gnu/libscotcherr-6.so (0x00007f307104c000)
    liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f3071021000)

and for libblas and libpapack. The output is:

aj@ajIIT:~/drake$ file /etc/alternatives/libblas.so.3-x86_64-linux-gnu 
/etc/alternatives/libblas.so.3-x86_64-linux-gnu: symbolic link to /usr/lib/x86_64-linux-gnu/atlas/libblas.so.3
aj@ajIIT:~/drake$ file /etc/alternatives/liblapack.so.3-x86_64-linux-gnu 
/etc/alternatives/liblapack.so.3-x86_64-linux-gnu: symbolic link to /usr/lib/x86_64-linux-gnu/atlas/liblapack.so.3

Also, when I run the bazel test. It says tolerance exceeds. Can this be a root cause? 

[       OK ] TestSOS.SimpleSos1 (1 ms)
[ RUN      ] TestSOS.MotzkinPolynomial
solvers/test/sos_examples.cc:109: Failure
Value of: symbolic::test::PolynomialEqual( m_result * r_result, monomial_basis2_.dot(gram2_val * monomial_basis2_), tol)
  Actual: false (The coefficient for x(0)^4 * x(0)^2 is -1.6422632143076044e-08, exceed tolerance 1.4999999999999999e-08
)
Expected: true
[  FAILED  ] TestSOS.MotzkinPolynomial (7 ms)
[ RUN      ] TestSOS.UnivariateNonnegative1
[       OK ] TestSOS.UnivariateNonnegative1 (1 ms)
[----------] 5 tests from TestSOS (12 ms total)
jwnimmer-tri commented 1 year ago

Okay, yes that explains it.

Your computer is using Atlas BLAS. Drake's default for all of our developers and our Continuous Integration testing is to use Netlib BLAS. Both versions of BLAS work basically the same, but might produce slightly different floating-point results depending on their order of operations.

On Ubuntu, which BLAS to use is an end-user configuration option, which Drake obeys. (There are several more BLAS options beyond just those two libraries, e.g., Intel MKL.) In some cases this means that our unit tests can produce false positives due to floating point tolerances.

You should be OK to stick with Atlas BLAS and just ignore this failure. You might get slightly different floating-point results versus other Drake users, but this can occur for many reasons already (e.g., macOS 12 vs Ubuntu 20.04 users).

If you want to stop using Atlas BLAS entirely, the Ubuntu command to remove it is something like sudo apt remove libatlas3-base (and/or possibly installing libblas-dev first). Note that that is a system-wide change, not just for Drake, so be cautious.

You can double-check which BLAS is being used via the file /etc/... commands we used above.

ajaygunalan commented 1 year ago

Thanks, it is solved by sudo apt remove libatlas3-base