MATLAB crashing while attempting to run exampleSlidingMass.m from Ubuntu 18.04 singularity container

[jshelly]

I created a singularity container with Ubuntu 18.04 and installed OpenSim (main branch, currently version 4.5) with Java bindings and Moco. Java (openjdk) version is 1.8.0_362. When trying to run exampleSlidingMass, it crashes with a segmentation fault each time it gets to solution = study.solve();. In this case, I have installed MATLAB (R2022b) directly into the container. I previously attempted using an external MATLAB bind mounted to the container. I had the same problems with the bind mount and hoped that installing MATLAB directly into the container would avoid the issues. Unfortunately, that was not the case. I also tried Ubuntu 20.04 with similar results. Matlab crash_dump files are included below:

matlab_crash_dump.3020669-1.txt matlab_crash_dump.3020669-2.txt

I've been reading through the forum and have seen many other users having library problems when using MATLAB with OpenSim Linux installations. My errors are similar to those reported by others, but not identical. I've tried some suggestions, with no success. Do you have any advice or is there any additional information that I could provide that might be useful?

Thanks, Shelly

[halleysfifthinc]

This is issue #2928; unfortunately, there is currently no solution/work around for this.

[aymanhab]

While I have no environment to troubleshoot this issue, the stack trace suggests that the crash happens in mkl library provided by Matlab. My guess would be that opensim was built against a different version of these libraries (Mkl includes Blas and lapack). If you can force the build system to use matlab mkl that may solve the problem. Please let us know what you find out.

On Mon, Mar 13, 2023, 7:46 AM Allen Hill @.***> wrote:

This is issue #2928 https://github.com/opensim-org/opensim-core/issues/2928; unfortunately, there is currently no solution/work around for this.

— Reply to this email directly, view it on GitHub https://github.com/opensim-org/opensim-core/issues/3423#issuecomment-1466280701, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA6JY4HZTVC7WFLMNZKBD33W34XLZANCNFSM6AAAAAAVYQY37A . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[jshelly]

Thanks for your responses @halleysfifthinc and @aymanhab. I did find an option to specify -DBUILD_USING_OTHER_LAPACK which I set as: -DBUILD_USING_OTHER_LAPACK:PATH="/usr/local/matlab/bin/glnxa64/mkl.so".

This worked for a while and eventually failed with:

[100%] Linking CXX executable ../../../simbody-visualizer
../../../libSimTKcommon.so.3.8: undefined reference to `cgetrf_'
../../../libSimTKcommon.so.3.8: undefined reference to `dgetri_'
../../../libSimTKcommon.so.3.8: undefined reference to `cgetri_'
../../../libSimTKcommon.so.3.8: undefined reference to `zgetri_'
../../../libSimTKcommon.so.3.8: undefined reference to `dgetrf_'
../../../libSimTKcommon.so.3.8: undefined reference to `zgetrf_'
../../../libSimTKcommon.so.3.8: undefined reference to `sgetrf_'
../../../libSimTKcommon.so.3.8: undefined reference to `sgetri_'
collect2: error: ld returned 1 exit status
Simbody/Visualizer/simbody-visualizer/CMakeFiles/simbody-visualizer.dir/build.make:120: recipe for target 'simbody-visualizer' failed
make[5]: *** [simbody-visualizer] Error 1
CMakeFiles/Makefile2:555: recipe for target 'Simbody/Visualizer/simbody-visualizer/CMakeFiles/simbody-visualizer.dir/all' failed
make[4]: *** [Simbody/Visualizer/simbody-visualizer/CMakeFiles/simbody-visualizer.dir/all] Error 2
Makefile:135: recipe for target 'all' failed
make[3]: *** [all] Error 2
CMakeFiles/simbody.dir/build.make:85: recipe for target 'simbody/stamp/simbody-build' failed
make[2]: *** [simbody/stamp/simbody-build] Error 2
CMakeFiles/Makefile2:124: recipe for target 'CMakeFiles/simbody.dir/all' failed
make[1]: *** [CMakeFiles/simbody.dir/all] Error 2
Makefile:90: recipe for target 'all' failed
make: *** [all] Error 2
FATAL:   While performing build: while running engine: exit status 2

I'm not sure how to overcome the undefined references. It doesn't seem happy with the matlab mkl library, or maybe there is another setting I am missing.

[jshelly]

Ah, in fact, looking further back, it seems the build option did not work at all:

checking whether build environment is sane... -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Could NOT find BLAS (missing: BLAS_LIBRARIES)
-- Could NOT find BLAS (missing: BLAS_LIBRARIES)
-- Could NOT find LAPACK (missing: LAPACK_LIBRARIES)
    Reason given by package: LAPACK could not be found because dependency BLAS could not be found.

CMake Warning at CMakeLists.txt:543 (message):
  Could not find blas/lapack

I got the format of the option from a README file, with the example:

-DBUILD_USING_OTHER_LAPACK:PATH="C:\Program Files\lapack-3.5.0\bin\liblapack.dll;C:\Program Files\lapack-3.5.0\bin\libblas.dll

I wonder if the form of that option isn't quite right.

[halleysfifthinc]

BUILD_USING_OTHER_LAPACK should be given as a linking command option, e.g. -L/usr/local/matlab/bin/glnxa64 -lmkl might work.

[jshelly]

Ah, ok. I'll give that a try. Thanks, @halleysfifthinc!

[jshelly]

That did not work either. I think the problem is that multiple dependencies require BLAS/LAPACK, and it seems they may have their own way(s) of specifying which libraries to use. I tried several different approaches. Unfortunately, none were successful, and I'm afraid I have spent as much time as I can afford to (and then some) on this problem. I have been working on this on behalf of another user at my university, in order to make it possible for them to use OpenSim our HPC system. For now, they will have to modify their workflow for use with the Python API, and hopefully there will be a solution further down the road.

Thanks again for all the suggestions!

Best, Shelly

[aymanhab]

This likely requires some digging into Matlab documentation of how they link Matlab to lapack and if it's possible to modify Simbody to link to their version or find why the issue happens only in some cases (I suspect it has to do with tables/template handling). Will revisit when time permits.

[jshelly]

Thanks, Ayman!