search

key4hep / k4FWCore

Core Components for the Gaudi-based Key4hep Framework
Apache License 2.0
10 stars 26 forks source link

Link against GaudiAlgLib to work with Gaudi v38 #189

Closed tmadlener closed 3 months ago

tmadlener commented 5 months ago

BEGINRELEASENOTES

ENDRELEASENOTES

jmcarcell commented 5 months ago

GaudiAlgLib is what we want to get rid of in https://github.com/key4hep/k4FWCore/issues/114. On top of that, Gaudi v38 has been in the stack for a while and tests work fine, see the CI of any recent PR. I don't understand, what's the problem?

tmadlener commented 5 months ago

Than it might be an issue with my local stack and a newer (>38.2) version of Gaudi. Let me investigate a bit further. Also this is for the tests, I think, so a normal build might work if it doesn't build the tests.

tmadlener commented 5 months ago

It might also be a Ubuntu thing? If I build on Ubuntu against the current nightly (2024-04-15), I get the following linker error without this PR:

&& /usr/bin/g++ -fPIC -fdiagnostics-color=always   -shared  -o test/k4FWCoreTest/libk4FWCoreTestPlugins.so test/k4FWCoreTest/CMakeFiles/k4FWCoreTestPlugins.dir/src/components/ExampleEventHeaderConsumer.cpp.o test/k4FWCoreTest/CMakeFiles/k4FWCoreTestPlugins.dir/src/components/ExampleFunctionalConsumer.cpp.o test/k4FWCoreTest/CMakeFiles/k4FWCoreTestPlugins.dir/src/components/ExampleFunctionalConsumerMultiple.cpp.o test/k4FWCoreTest/CMakeFiles/k4FWCoreTestPlugins.dir/src/components/ExampleFunctionalProducer.cpp.o test/k4FWCoreTest/CMakeFiles/k4FWCoreTestPlugins.dir/src/components/ExampleFunctionalProducerMultiple.cpp.o test/k4FWCoreTest/CMakeFiles/k4FWCoreTestPlugins.dir/src/components/ExampleFunctionalTransformer.cpp.o test/k4FWCoreTest/CMakeFiles/k4FWCoreTestPlugins.dir/src/components/ExampleFunctionalTransformerMultiple.cpp.o test/k4FWCoreTest/CMakeFiles/k4FWCoreTestPlugins.dir/src/components/ReadDDSimOutput.cpp.o test/k4FWCoreTest/CMakeFiles/k4FWCoreTestPlugins.dir/src/components/TestUniqueIDGenSvc.cpp.o test/k4FWCoreTest/CMakeFiles/k4FWCoreTestPlugins.dir/src/components/k4FWCoreTest_AlgorithmWithTFile.cpp.o test/k4FWCoreTest/CMakeFiles/k4FWCoreTestPlugins.dir/src/components/k4FWCoreTest_CheckExampleEventData.cpp.o test/k4FWCoreTest/CMakeFiles/k4FWCoreTestPlugins.dir/src/components/k4FWCoreTest_CreateExampleEventData.cpp.o test/k4FWCoreTest/CMakeFiles/k4FWCoreTestPlugins.dir/src/components/k4FWCoreTest_HelloWorldAlg.cpp.o test/k4FWCoreTest/CMakeFiles/k4FWCoreTestPlugins.dir/src/components/k4FWCoreTest_cellID_reader.cpp.o test/k4FWCoreTest/CMakeFiles/k4FWCoreTestPlugins.dir/src/components/k4FWCoreTest_cellID_writer.cpp.o -L/home/tmadlener/work/key4hep/k4FWCore/build/.plugins -Wl,-rpath,/home/tmadlener/work/key4hep/k4FWCore/build/.plugins:/home/tmadlener/work/key4hep/k4FWCore/build/k4FWCore:/cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-04-15/x86_64-ubuntu22.04-gcc11.4.0-opt/edm4hep/792cefc3a0f7cb3dfa12aba50a24a6f37089e84e_develop-v2np7t/lib:/cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/gaudi/38.0-76wkel/lib:/cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/boost/1.84.0-vbjlqy/lib:/cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/intel-tbb/2021.9.0-frifn6/lib:/cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/python/3.10.13-7ljnw7/lib:/cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-04-15/x86_64-ubuntu22.04-gcc11.4.0-opt/podio/428150fe3704867ae0b24b5df5b7a0fe803681a9_develop-hw3hk6/lib:/cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/root/6.28.10-or4q4m/lib/root:/cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/vdt/0.4.4-eq3nlo/lib:  k4FWCore/libk4FWCore.so  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-04-15/x86_64-ubuntu22.04-gcc11.4.0-opt/edm4hep/792cefc3a0f7cb3dfa12aba50a24a6f37089e84e_develop-v2np7t/lib/libedm4hepDict.so  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-04-15/x86_64-ubuntu22.04-gcc11.4.0-opt/edm4hep/792cefc3a0f7cb3dfa12aba50a24a6f37089e84e_develop-v2np7t/lib/libedm4hep.so  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/gaudi/38.0-76wkel/lib/libGaudiKernel.so  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/gaudi/38.0-76wkel/lib/libGaudiPluginService.so  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/boost/1.84.0-vbjlqy/lib/libboost_filesystem.so.1.84.0  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/boost/1.84.0-vbjlqy/lib/libboost_atomic.so.1.84.0  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/boost/1.84.0-vbjlqy/lib/libboost_thread.so.1.84.0  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/boost/1.84.0-vbjlqy/lib/libboost_regex.so.1.84.0  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/boost/1.84.0-vbjlqy/lib/libboost_system.so.1.84.0  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/intel-tbb/2021.9.0-frifn6/lib/libtbb.so.12.9  -ldl  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/fmt/10.2.1-ysbeqp/lib/libfmt.a  /usr/lib/x86_64-linux-gnu/librt.a  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-04-15/x86_64-ubuntu22.04-gcc11.4.0-opt/podio/428150fe3704867ae0b24b5df5b7a0fe803681a9_develop-hw3hk6/lib/libpodioRootIO.so  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/root/6.28.10-or4q4m/lib/root/libTree.so.6.28.10  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/root/6.28.10-or4q4m/lib/root/libMathCore.so.6.28.10  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/root/6.28.10-or4q4m/lib/root/libROOTNTuple.so.6.28.10  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/root/6.28.10-or4q4m/lib/root/libImt.so.6.28.10  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/root/6.28.10-or4q4m/lib/root/libMultiProc.so.6.28.10  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/root/6.28.10-or4q4m/lib/root/libNet.so.6.28.10  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/root/6.28.10-or4q4m/lib/root/libRIO.so.6.28.10  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/root/6.28.10-or4q4m/lib/root/libThread.so.6.28.10  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/root/6.28.10-or4q4m/lib/root/libROOTVecOps.so.6.28.10  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/vdt/0.4.4-eq3nlo/lib/libvdt.so  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/root/6.28.10-or4q4m/lib/root/libCore.so.6.28.10  /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-04-15/x86_64-ubuntu22.04-gcc11.4.0-opt/podio/428150fe3704867ae0b24b5df5b7a0fe803681a9_develop-hw3hk6/lib/libpodio.so  -Wl,-rpath-link,/cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/python/3.10.13-7ljnw7/lib && cd /home/tmadlener/work/key4hep/k4FWCore/build/test/k4FWCoreTest && /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/cmake/3.27.9-u6cj5k/bin/cmake -E create_symlink /home/tmadlener/work/key4hep/k4FWCore/build/test/k4FWCoreTest/libk4FWCoreTestPlugins.so /home/tmadlener/work/key4hep/k4FWCore/build/.plugins/libk4FWCoreTestPlugins.so && cd /home/tmadlener/work/key4hep/k4FWCore/build/test/k4FWCoreTest && /home/tmadlener/work/key4hep/k4FWCore/build/run /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/gaudi/38.0-76wkel/bin/listcomponents --output k4FWCoreTestPlugins.components libk4FWCoreTestPlugins.so && cd /home/tmadlener/work/key4hep/k4FWCore/build/test/k4FWCoreTest && /home/tmadlener/work/key4hep/k4FWCore/build/run /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2024-03-08/x86_64-ubuntu22.04-gcc11.4.0-opt/gaudi/38.0-76wkel/bin/genconf --configurable-module=GaudiKernel.Proxy --configurable-default-name=Configurable.DefaultName --configurable-algorithm=ConfigurableAlgorithm --configurable-algtool=ConfigurableAlgTool --configurable-auditor=ConfigurableAuditor --configurable-service=ConfigurableService -o genConfDir/k4FWCoreTest -p k4FWCoreTest -i /home/tmadlener/work/key4hep/k4FWCore/build/test/k4FWCoreTest/libk4FWCoreTestPlugins.so
ERROR: failed to load libk4FWCoreTestPlugins.so: /home/tmadlener/work/key4hep/k4FWCore/build/test/k4FWCoreTest/libk4FWCoreTestPlugins.so: undefined symbol: _ZN14GaudiAlgorithm10sysExecuteERK12EventContext
[26/29] Linking CXX shared module k4FWCore/libk4FWCorePlugins.so

with

$ c++filt _ZN14GaudiAlgorithm10sysExecuteERK12EventContext
GaudiAlgorithm::sysExecute(EventContext const&)

I have no explanation however, how spack is able to build it in this case.

Edit: I also fail to build k4FWCore with spack locally against gaudi@master without this and see the same linker error.

As far as I can tell I build gaudi the same way as in the nightlies:

gaudi@master+aida+cppunit~docs+examples+gaudialg+gperftools+heppdt~ipo+jemalloc+unwind~vtune+xercesc
tmadlener commented 4 months ago

I seem to require this locally for the later versions of Gaudi. Given that it's only the test library that needs this, I would be in favor of having this in. Alternatively, we get rid of the necessity for GaudiAlg in the tests, but that will potentially take a bit longer.

jmcarcell commented 3 months ago

Now all the references to GaudiAlg should be gone for this repository

tmadlener commented 3 months ago

Now I run into a undefined symbol: _ZTI14GaudiAlgorithm. I am not really sure what is different on my end and for the nightlies which are obviously working.

Edit: At least for my build that symbol is definitely defined in libGaudiAlgLib.so:

$readelf -s -w <prefix>/lib/libGaudiAlgLib.so | grep _ZTI14GaudiAlgorithm
   799: 000000000025fb70    24 OBJECT  WEAK   DEFAULT   22 _ZTI14GaudiAlgorithm
  1661: 000000000025fb70    24 OBJECT  WEAK   DEFAULT   22 _ZTI14GaudiAlgorithm

I am on this commit for Gaudi.

Edit 2: I also cannot build k4FWCore against the nightlies on Ubuntu 22.04. This most likely makes something on my local setup the cause of this as the nightlies for Ubuntu are working here in CI.

jmcarcell commented 3 months ago

Is it when linking or running some dictionary generation?

tmadlener commented 3 months ago

It's during linking. I have no idea why it fails though. The library contents for Gaudi are the same for the nightlies and for me locally. However for the stacks deployed on CVMFS libk4FWCoreTestPlugins.so (which is the one that fails linking for me), does not depend on libGaudiAlgLib.so at all. It looks like the linker (settings?) is trying to pull something in that it does not necessarily need. Anyhow, I have a local workaround for now, and it doesn't seem to happen for the nightlies, so we can keep this on low priority.

andresailer commented 3 months ago

Not reproducible