Cling cannot load libDDG4Plugins

StrahinjaLukic commented 6 years ago

I am trying to run ddsim from v01-19-05 and CLIC detector. I am getting the following runtime error:

ddsim --steeringFile clic_steer.py --compactFile ../xml/CLIC_o2_v05/CLIC_o2_v05.xml --gun.particle "pi+"
cling::DynamicLibraryManager::loadLibrary(): dlopen: cannot load any more object with static TLS
+--------------------------------------------------------------------------------------------------------+
|  Failed to load DDG4 library:                                                                          |
|  DDG4.py: Failed to load the DDG4 library libDDG4Plugins: No such file or directory                    |
+--------------------------------------------------------------------------------------------------------+

However, libDDG4Plugins.so is located at the expected place:

ll $DD4HEP/lib/libDDG4Plugins*
-rw-r--r-- 1 strahinja strahinja    12008 нов 24 15:50 /nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/DD4hep/v01-05/lib/libDDG4Plugins.components
lrwxrwxrwx 1 strahinja strahinja       21 нов 24 15:52 /nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/DD4hep/v01-05/lib/libDDG4Plugins.so -> libDDG4Plugins.so.1.5
-rw-r--r-- 1 strahinja strahinja 10112985 нов 24 15:50 /nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/DD4hep/v01-05/lib/libDDG4Plugins.so.1.5

Has anyone had a similar problem before? Is there a way I can check where cling::DynamicLibraryManager is looking for libraries?

OS version: Ubuntu linux 14.04
Compiler version: GCC 6.4
Package version: tag v01-19-05
Reproduced by: install ilcSoft using iLCInstall (tag v01-19-05), source init_ilcsoft.sh of the installation, add GCC 6.4 to PATH, run ddsim command like above.
Input: clic_steer.py is from master branch on github: link to clic_steer.py I made the following changes in clic_steer.py:
```
SIM.enableGun = True
SIM.numberOfEvents = 3
```
CLIC_o2_v05.xml is from tag v01-19-05.
Output: full build and run log output release-ilcsoft-v01-19-05-2017-11-24_20-29.log For the full run log see above.
Goal: run ddsim with particle gun and CLIC detector.

andresailer commented 6 years ago

The real problem is

cling::DynamicLibraryManager::loadLibrary(): dlopen: cannot load any more object with static TLS

which is a major pain.

andresailer commented 6 years ago

What is the output of

ldd /nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/DD4hep/v01-05/lib/libDDG4Plugins.so.1.5

StrahinjaLukic commented 6 years ago

Attaching output of ldd /nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/DD4hep/v01-05/lib/libDDG4Plugins.so.1.5: lddoutput.txt

andresailer commented 6 years ago

please run

python -c "import ROOT; print ROOT.gSystem.Load('libglapi')"
python -c "import ROOT; print ROOT.gSystem.Load('libDDG4Plugins')"
python -c "import ROOT; print ROOT.gSystem.Load('libglapi'); print ROOT.gSystem.Load('libDDG4Plugins')"

and (when loading the dd4hep python we load libglapi first, because that reserves a larger space for thread local storage (TLS)), maybe that doesn't work Also the output of, because it might not pick up the correct library if there are two versions 32/64 bit

ls -l /usr/lib/x86_64-linux-gnu/libglapi*
locate libglapi
echo $LD_LIBRARY_PATH

StrahinjaLukic commented 6 years ago

python -c "import ROOT; print ROOT.gSystem.Load('libglapi')"
Error in <TUnixSystem::FindDynamicLibrary>: libglapi[.so | .dll | .dylib | .sl | .dl | .a] does not exist in /nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/geant4/10.03.p02/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/ilcutil/v01-03/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/mysql/usr/lib64:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/mysql/usr/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/mysql/usr/lib64/mysql:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/mysql/usr/lib/mysql:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/DD4hepExamples/v01-05/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/lcgeo/v00-15-03/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/FastJet/3.2.0/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/CEDViewer/v01-15/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/LCFIVertex/v00-07-04/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/PandoraAnalysis/v02-00-00/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/PandoraPFANew/v03-06-00/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/CED/v01-09-02/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/MarlinUtil/v01-14/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/GBL/V02-01-01/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/gsl/2.1/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/aidaTT/v00-08/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/KalTest/v02-03/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/xercesc/3.2.0/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/QT/4.8.5/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/DD4hep/v01-05/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/gear/v01-07/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/CLHEP/2.3.4.3/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/root/6.08.06/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/lcio/v02-11/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/CondDBMySQL/CondDBMySQL_ILC-0-9-6/lib:/nfs_SL/software/gcc/6.4.0/lib64:/nfs_SL/software/gcc/6.4.0/lib:/usr/lib:/nfs_SL/software/gcc/6.4.0/lib64:/nfs_SL/ilcsoft/x86_64_gcc64_ub1404/CLHEP/2.3.4.3/lib:/nfs_SL/ilcsoft/x86_64_gcc64_ub1404/xercesc/3.2.0/lib:.:/nfs_SL/ilcsoft/x86_64_gcc64_ub1404/root/6.08.06/lib:/usr/lib/tls/x86_64:/usr/lib/tls:/usr/lib/x86_64:/usr/lib
-1

I notice that python is not looking in /usr/lib/x86_64-linux-gnu/

StrahinjaLukic commented 6 years ago

python -c "import ROOT; print ROOT.gSystem.Load('libDDG4Plugins')"
cling::DynamicLibraryManager::loadLibrary(): dlopen: cannot load any more object with static TLS
-1

StrahinjaLukic commented 6 years ago

python -c "import ROOT; print ROOT.gSystem.Load('libglapi'); print ROOT.gSystem.Load('libDDG4Plugins')"
Error in <TUnixSystem::FindDynamicLibrary>: libglapi[.so | .dll | .dylib | .sl | .dl | .a] does not exist in /nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/geant4/10.03.p02/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/ilcutil/v01-03/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/mysql/usr/lib64:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/mysql/usr/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/mysql/usr/lib64/mysql:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/mysql/usr/lib/mysql:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/DD4hepExamples/v01-05/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/lcgeo/v00-15-03/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/FastJet/3.2.0/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/CEDViewer/v01-15/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/LCFIVertex/v00-07-04/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/PandoraAnalysis/v02-00-00/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/PandoraPFANew/v03-06-00/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/CED/v01-09-02/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/MarlinUtil/v01-14/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/GBL/V02-01-01/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/gsl/2.1/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/aidaTT/v00-08/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/KalTest/v02-03/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/xercesc/3.2.0/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/QT/4.8.5/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/DD4hep/v01-05/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/gear/v01-07/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/CLHEP/2.3.4.3/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/root/6.08.06/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/lcio/v02-11/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/CondDBMySQL/CondDBMySQL_ILC-0-9-6/lib:/nfs_SL/software/gcc/6.4.0/lib64:/nfs_SL/software/gcc/6.4.0/lib:/usr/lib:/nfs_SL/software/gcc/6.4.0/lib64:/nfs_SL/ilcsoft/x86_64_gcc64_ub1404/CLHEP/2.3.4.3/lib:/nfs_SL/ilcsoft/x86_64_gcc64_ub1404/xercesc/3.2.0/lib:.:/nfs_SL/ilcsoft/x86_64_gcc64_ub1404/root/6.08.06/lib:/usr/lib/tls/x86_64:/usr/lib/tls:/usr/lib/x86_64:/usr/lib
-1
cling::DynamicLibraryManager::loadLibrary(): dlopen: cannot load any more object with static TLS
-1

StrahinjaLukic commented 6 years ago

ls -l /usr/lib/x86_64-linux-gnu/libglapi*
lrwxrwxrwx 1 root root     17 јан 12  2016 /usr/lib/x86_64-linux-gnu/libglapi.so -> libglapi.so.0.0.0
lrwxrwxrwx 1 root root     17 јул 18 21:00 /usr/lib/x86_64-linux-gnu/libglapi.so.0 -> libglapi.so.0.0.0
-rw-r--r-- 1 root root 182528 јул 18 21:01 /usr/lib/x86_64-linux-gnu/libglapi.so.0.0.0

StrahinjaLukic commented 6 years ago

locate libglapi
/usr/lib/i386-linux-gnu/libglapi.so.0
/usr/lib/i386-linux-gnu/libglapi.so.0.0.0
/usr/lib/x86_64-linux-gnu/libglapi.so
/usr/lib/x86_64-linux-gnu/libglapi.so.0
/usr/lib/x86_64-linux-gnu/libglapi.so.0.0.0
/usr/share/bug/libglapi-mesa-lts-xenial
/usr/share/bug/libglapi-mesa-lts-xenial/control
/usr/share/bug/libglapi-mesa-lts-xenial/script
/usr/share/doc/libglapi-mesa-lts-xenial
/usr/share/doc/libglapi-mesa-lts-xenial/changelog.Debian.gz
/usr/share/doc/libglapi-mesa-lts-xenial/copyright
/usr/share/lintian/overrides/libglapi-mesa-lts-xenial
/var/lib/dpkg/info/libglapi-mesa-lts-xenial:amd64.list
/var/lib/dpkg/info/libglapi-mesa-lts-xenial:amd64.md5sums
/var/lib/dpkg/info/libglapi-mesa-lts-xenial:amd64.postinst
/var/lib/dpkg/info/libglapi-mesa-lts-xenial:amd64.postrm
/var/lib/dpkg/info/libglapi-mesa-lts-xenial:amd64.shlibs
/var/lib/dpkg/info/libglapi-mesa-lts-xenial:i386.list
/var/lib/dpkg/info/libglapi-mesa-lts-xenial:i386.md5sums
/var/lib/dpkg/info/libglapi-mesa-lts-xenial:i386.postinst
/var/lib/dpkg/info/libglapi-mesa-lts-xenial:i386.postrm
/var/lib/dpkg/info/libglapi-mesa-lts-xenial:i386.shlibs

StrahinjaLukic commented 6 years ago

echo $LD_LIBRARY_PATH
/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/geant4/10.03.p02/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/ilcutil/v01-03/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/mysql/usr/lib64:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/mysql/usr/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/mysql/usr/lib64/mysql:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/mysql/usr/lib/mysql:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/DD4hepExamples/v01-05/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/lcgeo/v00-15-03/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/FastJet/3.2.0/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/CEDViewer/v01-15/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/LCFIVertex/v00-07-04/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/PandoraAnalysis/v02-00-00/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/PandoraPFANew/v03-06-00/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/CED/v01-09-02/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/MarlinUtil/v01-14/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/GBL/V02-01-01/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/gsl/2.1/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/aidaTT/v00-08/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/KalTest/v02-03/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/xercesc/3.2.0/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/QT/4.8.5/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/DD4hep/v01-05/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/gear/v01-07/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/CLHEP/2.3.4.3/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/root/6.08.06/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/lcio/v02-11/lib:/nfs_SL/ilcsoft/builds/x86_64_gcc64_ub1404/v01-19-05/CondDBMySQL/CondDBMySQL_ILC-0-9-6/lib:/nfs_SL/software/gcc/6.4.0/lib64:/nfs_SL/software/gcc/6.4.0/lib:/usr/lib:/nfs_SL/software/gcc/6.4.0/lib64::/nfs_SL/ilcsoft/x86_64_gcc64_ub1404/CLHEP/2.3.4.3/lib:/nfs_SL/ilcsoft/x86_64_gcc64_ub1404/xercesc/3.2.0/lib

andresailer commented 6 years ago

please add /usr/lib/x86_64-linux-gnu/ to the LD_LIBRARY_PATH and try

python -c "import ROOT; print ROOT.gSystem.Load('libglapi'); print ROOT.gSystem.Load('libDDG4Plugins')"

again, and if that works, try ddsim

StrahinjaLukic commented 6 years ago

Yes! I had to add /usr/lib/x86_64-linux-gnu/ to the end of the LD_LIBRARY_PATH so that libstdc++.so.6 is loaded from the GCC 6.4.0 location. Now the output of the python imports is

python -c "import ROOT; print ROOT.gSystem.Load('libglapi'); print ROOT.gSystem.Load('libDDG4Plugins')"
0
0

And ddsim works. Thanks!

iLCSoft / CLICPerformance

Cling cannot load libDDG4Plugins #52