Cannot build on debian 11 anymore

SietseAchterop commented 1 year ago

Currently I cannot build opensim-gui from source on debian 11 anymore. I am using openjdk-8 from debian/sid. Half a year ago it worked fine. I use the master branch on github and a script I derived from CI-script. It is effectively the same as the current script in the documentation.

The build of the dependencies and core goes just fine, but the build of the opensim-gui part fails in the last step. I do the following:

cd ${OPENSIM_DIR}/build
cmake  ../opensim-gui -DCMAKE_PREFIX_PATH=${OPENSIM_DIR}/opensim-core-install -DANT_ARGS="-Dnbplatform.default.netbeans.dest.dir=/usr/local/netbeans-12.3/netbeans;-Dnbplatform.default.harness.dir=/usr/local/netbeans-12.3/netbeans/harness"
make CopyOpenSimCore
make PrepareInstaller

The error occurs in the make PrepareInstaller, the last part of the log is:

-do-compile:
   [depend] Deleted 650 out of date files in 0 seconds
 [nb-javac] Compiling 703 source files to /home/sietse/Software/OpenSim/opensim-gui/Gui/opensim/modeling/build/classes
 [nb-javac] warning: [options] bootstrap class path not set in conjunction with -source 1.6
 [nb-javac] /home/sietse/Software/OpenSim/opensim-gui/Gui/opensim/modeling/src/org/opensim/modeling/SharedOrientationsReference.java:33: error: cannot find symbol
 [nb-javac]         opensimSimulationJNI.delete_SharedOrientationsReference(swigCPtr);
 [nb-javac]                             ^
 [nb-javac]   symbol:   method delete_SharedOrientationsReference(long)
 [nb-javac]   location: class opensimSimulationJNI
 [nb-javac] /home/sietse/Software/OpenSim/opensim-gui/Gui/opensim/modeling/src/org/opensim/modeling/SharedOrientationsReference.java:40: error: cannot find symbol
 [nb-javac]     this(opensimSimulationJNI.new_SharedOrientationsReference(), true);
 [nb-javac]                              ^
 [nb-javac]   symbol:   method new_SharedOrientationsReference()
 [nb-javac]   location: class opensimSimulationJNI
 [nb-javac] 2 errors
 [nb-javac] 1 warning

BUILD FAILED
/usr/local/netbeans-12.3/netbeans/harness/suite.xml:163: The following error occurred while executing this line:
/usr/local/netbeans-12.3/netbeans/harness/common.xml:206: Compile failed; see the compiler error output for details.

Total time: 5 seconds
make[3]: *** [Gui/opensim/CMakeFiles/PrepareInstaller.dir/build.make:77: Gui/opensim/CMakeFiles/PrepareInstaller] Error 1
make[2]: *** [CMakeFiles/Makefile2:148: Gui/opensim/CMakeFiles/PrepareInstaller.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:155: Gui/opensim/CMakeFiles/PrepareInstaller.dir/rule] Error 2
make: *** [Makefile:150: PrepareInstaller] Error 2

Why can opensimSimulationJNI not be found? The class file is created in the build_core directory:

../opensim-core-install/sdk/Java/org/opensim/modeling/opensimSimulationJNI.java

What could be the problem here? Thanks in advance, Sietse

AlbertoCasasOrtiz commented 1 year ago

Hello @SietseAchterop, could you share the script you are using so I can replicate the issue on Debian 11?

We have now new scripts in the wiki. The new scritps use Temurin JDK 8. These scripts are tested on Debian 10 and 11, and should work fine. If you use them, please, let me know if you find any issue.

SietseAchterop commented 1 year ago

Thanks for the reply Alberto!

Please find my script at the bottom of this message. It does mostly the same as your script. In my script I use make instead of ninja, but the most important difference is that I also make the opensim-gui program. And the error that I mentioned earlier is about that last part. It tested your script and it works just fine here. I also tested your script with make and mine with ninja. Doesn't make any difference.

In the end I completely reinstalled my linux machine because it had some history, so I thought that that could cause problems. But the same problems occur on the freshly installed debian 11 I am using now.

I added the gui-part to your script and it also fails, but strangely in a different way than in my script. Strange because the cmake/make calls are almost the same in both situations. I added the following code at the and of your script:

# Get opensim-gui
echo "LOG: CLONING OPENSIM-GUI..."
git -C ~/opensim-workspace/opensim-gui-source pull || git clone https://github.com/opensim-org/opensim-gui.git ~/opensim-workspace/opensim-gui-source
echo

# Build opensim-gui.
echo "LOG: BUILDING OPENSIM-GUI..."
mkdir -p ~/opensim-workspace/opensim-gui-build || true
cd ~/opensim-workspace/opensim-gui-build
cmake  ~/opensim-workspace/opensim-gui-source -DCMAKE_PREFIX_PATH=~/opensim-core -DANT_ARGS="-Dnbplatform.default.netbeans.dest.dir=/usr/local/netbeans-13/netbeans;-Dnbplatform.default.harness.dir=/usr/local/netbeans-13/netbeans/harness"
# dit 
make CopyOpenSimCore
make PrepareInstaller

It also fails in the last make. Here the output of the above addition:

LOG: CLONING OPENSIM-GUI...
fatal: cannot change to '/home/sietse/opensim-workspace/opensim-gui-source': No such file or directory
Cloning into '/home/sietse/opensim-workspace/opensim-gui-source'...
remote: Enumerating objects: 48396, done.
remote: Counting objects: 100% (1944/1944), done.
remote: Compressing objects: 100% (691/691), done.
remote: Total 48396 (delta 1198), reused 1784 (delta 1086), pack-reused 46452
Receiving objects: 100% (48396/48396), 325.67 MiB | 21.79 MiB/s, done.
Resolving deltas: 100% (31594/31594), done.

LOG: BUILDING OPENSIM-GUI...
-- The C compiler identification is GNU 10.2.1
-- The CXX compiler identification is GNU 10.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found Ant: /usr/bin/ant  
-- Found Git: /usr/bin/git (found version "2.30.2") 
-- Build VERSION: 4.4-2022-11-22-3928a644
-- Configuring done
-- Generating done
-- Build files have been written to: /home/sietse/opensim-workspace/opensim-gui-build
Scanning dependencies of target CopyOpenSimCore
[100%] Copy opensim-core Java Bindings to GUI.
Buildfile: /home/sietse/opensim-workspace/opensim-gui-source/Gui/opensim/build.xml

copy-java-bindings:
     [copy] Copying 852 files to /home/sietse/opensim-workspace/opensim-gui-source/Gui/opensim/modeling/src/org/opensim/modeling

BUILD SUCCESSFUL
Total time: 0 seconds
[100%] Built target CopyOpenSimCore
Scanning dependencies of target opensim-models
[ 11%] Creating directories for 'opensim-models'
[ 22%] No download step for 'opensim-models'
[ 33%] No update step for 'opensim-models'
[ 44%] No patch step for 'opensim-models'
[ 55%] Performing configure step for 'opensim-models'
CMake Error: The source directory "/home/sietse/opensim-workspace/opensim-gui-source/opensim-models" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
make[3]: *** [Gui/opensim/CMakeFiles/opensim-models.dir/build.make:125: Gui/opensim/opensim-models-prefix/src/opensim-models-stamp/opensim-models-configure] Error 1
make[2]: *** [CMakeFiles/Makefile2:175: Gui/opensim/CMakeFiles/opensim-models.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:155: Gui/opensim/CMakeFiles/PrepareInstaller.dir/rule] Error 2
make: *** [Makefile:150: PrepareInstaller] Error 2

So a completely different type of error from what I mentioned earlier. What could be the problem here?

Before giving my script here a few remarks about your script to make it a little more robust.

About java It maybe is better to test whether the proper version of java is already installed. (I also tested with the debian/unstable version, which is actually the same version, and it makes no difference at all.) When java was already installed I do: update-java-alternatives -s temurin-8-jdk-amd64 but this is probably just an alternative to setting JAVA_HOME.
About swig Could also already be installed, so you could test for that. Version 4.0.2 is directly available in debian 11.
testing opensim-cmd I get: libadolc.so.2 not found This is an known problem. Somehow this symbolic link is not made. See also the end of my script.

Now finally my script to create opensim including the gui version. I use a adapted version of Simbody for my project, therefore the copying from Patches-opensim in this script. I have commented them out below. More about this project in: https://github.com/SietseAchterop/Rowing-simulator

I have the following in my .bashrc for using the opensim commands later: export OPENSIM_INSTALL_DIR=/home/sietse/Software/OpenSim export LD_LIBRARY_PATH=$OPENSIM_INSTALL_DIR/opensim/sdk/lib:~/.opensim/plugins:/usr/local/lib

============================================================== script =================

#!/bin/bash
#   Ad hoc script to build opensim from source

echo
echo "   ONLY execute this script once!"
echo "       After that use the make_nodep script that only does the part after building the dependencies."
echo
read -p    "Really continue?  (yes/no) " antwoord
if [ $antwoord != yes ]; then exit 0; fi

#   Assume that the following prerequisites are met
#     Installed the following
#       On debian 11:
#          sudo apt-get update && sudo apt-get install --yes liblapack-dev freeglut3-dev libxi-dev libxmu-dev doxygen python3 python3-dev python3-numpy python3-setuptools swig ant libopenblas-dev liblapack-dev
#       On Ubuntu 20.04
#          swig 4.0.2  has to be installed  --> /usr/local
#     Install and select java 1.8 from debian/ubuntu
#     Install Netbeans 12.5 from netbeans site.
#     Tested only with python 3.8 or 3.9 on Debian

# For the gui, make sure that the locale is en_US.UTF-8

# to test debian or ubuntu:
VERSION=`awk -F= '/^VERSION_ID/{print $2}' /etc/os-release`

# helps a bit.
set -e

# change this for a different location
OPENSIM_DIR=/home/sietse/Software/OpenSim
cd ${OPENSIM_DIR}

# to completely cleanup do:
#   rm -rf build  build_core  build_deps  build_simbody opensim  opensim-core  opensim-core-install  opensim_dependencies_install  opensim-gui

#     Get sources   
git clone             https://github.com/opensim-org/opensim-core.git
git clone --recursive https://github.com/opensim-org/opensim-gui.git

#   I install Simbody from source because of the changes for BladeForce
git clone             https://github.com/simbody/simbody.git

# Add patches for the BladeForce
#cp -r Patches-opensim/opensim-core .
#cp -r Patches-opensim/simbody .

# test for previous install
if [ -d ${OPENSIM_DIR}/opensim ]
then
    echo "Directory ${OPENSIM_DIR}/opensim exists on your filesystem."
    echo "   Please remove before continuing."
    exit 1
fi

cd ${OPENSIM_DIR}
if [ ! -d build ]
then
   mkdir build build_deps build_core build_simbody
fi

# We first build simbody, outside of opensim to make patching/rebuilding it easier.
export SIMBODY_HOME=/usr/local

#  some tests have to be removed before building
#   TestExponentialSpring.cpp
#       ....
#   all adhoc tests
cd build_simbody
cmake ../simbody
make -j8
sudo make install

# patched dependencies/CMakeLists.txt to force no build of simbody here
cd ${OPENSIM_DIR}/build_deps
cmake ../opensim-core/dependencies -DCMAKE_INSTALL_PREFIX=${OPENSIM_DIR}/opensim_dependencies_install -DSUPERBUILD_ezc3d=ON -DOPENSIM_WITH_TROPTER=ON -DOPENSIM_WITH_CASADI=ON -DSUPERBUILD_simbody=OFF
cmake . -LAH
cmake --build . --config Release

# the nodep script only does the part after this

cd ${OPENSIM_DIR}/build_core
cmake  ../opensim-core -DOPENSIM_DEPENDENCIES_DIR=${OPENSIM_DIR}/opensim_dependencies_install -DBUILD_JAVA_WRAPPING=on -DBUILD_PYTHON_WRAPPING=on -DOPENSIM_C3D_PARSER=ezc3d -DBUILD_TESTING=off -DCMAKE_INSTALL_PREFIX=${OPENSIM_DIR}/opensim-core-install -DOPENSIM_INSTALL_UNIX_FHS=OFF -DCMAKE_EXPORT_COMPILE_COMMANDS=1
cmake . -LAH
cmake --build . --config Release 
cmake --install .

cd ${OPENSIM_DIR}/build
cmake  ../opensim-gui -DCMAKE_PREFIX_PATH=${OPENSIM_DIR}/opensim-core-install -DANT_ARGS="-Dnbplatform.default.netbeans.dest.dir=/usr/local/netbeans-13/netbeans;-Dnbplatform.default.harness.dir=/usr/local/netbeans-13/netbeans/harness"
make CopyOpenSimCore
make PrepareInstaller

cd ${OPENSIM_DIR}
tar xf opensim-gui/Gui/opensim/dist/OpenSim-4.*
chmod 755 opensim/bin/*

# ad hoc
pushd opensim/sdk/lib
ln -s libadolc.so.2.1.0 libadolc.so.2
cp ../../../opensim_dependencies_install/ipopt/lib/libipopt.so.1.10.8 .
ln -s libipopt.so.1.10.8 libipopt.so.1
popd

echo ==== done  ====

SietseAchterop commented 1 year ago

I now basically have it working! An essential --recursive was missing in the script that I ran after your script. The qui now works, but there are a few strange things.

There are only 4 tools under the Tools menu item. Notably CMC is missing.
When I run an IK, it ends directly with the following error: [error] InverseKinematicsTool Failed: Timestamp at row 0 with value 0,000000 is greater-than/equal to timestamp at row 1 with value 0,000000 which is NOT the case. When I run it via "opensim-cmd run-tool" it works fine!

What could be the reason for there errors?

Below the script that I now use, it can be called after the script from the wiki.


#!/bin/bash
set -e
#  netbeans and ant should be installed.
#  executed after opensim-core-linux-build-script.sh
#   the last "cmake --install ."  is not needed anymore, the opensim dir will contain it all.

# Get opensim-gui
echo "LOG: CLONING OPENSIM-GUI..."
git -C ~/opensim-workspace/opensim-gui-source pull || git clone --recursive https://github.com/opensim-org/opensim-gui.git ~/opensim-workspace/opensim-gui-source
echo

# Build opensim-gui.
echo "LOG: BUILDING OPENSIM-GUI..."
mkdir -p ~/opensim-workspace/opensim-gui-build || true
cd ~/opensim-workspace/opensim-gui-build
cmake  ~/opensim-workspace/opensim-gui-source -DCMAKE_PREFIX_PATH=~/opensim-core -DANT_ARGS="-Dnbplatform.default.netbeans.dest.dir=/usr/local/netbeans-13/netbeans;-Dnbplatform.default.harness.dir=/usr/local/netbeans-13/netbeans/harness"
make CopyOpenSimCore
make PrepareInstaller

# Install opensim-core and opensim-gui in ~/opensim
echo "LOG: INSTALL OPENSIM-GUI..."
cd ~/
tar xf ~/opensim-workspace/opensim-gui-source/Gui/opensim/dist/OpenSim-4.*
# execution permissions were not set!
chmod 755 opensim/bin/*

# ad hoc
pushd opensim/sdk/lib
ln -s libadolc.so.2.1.0 libadolc.so.2
cp ../../../opensim-workspace/opensim-core-dependencies-install/ipopt/lib/libipopt.so.1.10.8 .
ln -s libipopt.so.1.10.8 libipopt.so.1
popd
}

aymanhab commented 1 year ago

The timestamp issue is related to using European locale when the code/parsing is expecting US (with "." for decimal point rather than ","). Not sure why some tools would show up while others don't when they all utilize same underlying machinery. Maybe error messages or warnings from build log would explain

SietseAchterop commented 1 year ago

Thanks! If have been bitten by that before. The problem is gone when setting all locale to en_US.UTF-8. Now there only is the problem of the missing Tool menu items.

AlbertoCasasOrtiz commented 1 year ago

@aymanhab I have been investigating the missing tools issue and looks like this goes beyond Debian. Some tools are missing on the latest Windows artifact from CI and on Ubuntu as well. I am going to open a new issue regarding the missing tools.

@SietseAchterop Do you need anything else regarding building on Debian 11? If not, please close this issue, and we will continue investigating the missing tools issue in a new one.

SietseAchterop commented 1 year ago

That's a good idea. I'll close the issue.

Note that I have the missing tools problem in both a newly installed ubuntu 22.04 and a newly installed debian 11.

opensim-org / opensim-gui

Cannot build on debian 11 anymore #1405