search

root-project / root

The official repository for ROOT: analyzing, storing and visualizing big data, scientifically
https://root.cern
Other
2.7k stars 1.28k forks source link

Instantiating a ROOT.vector adds the build directory to cling's include paths #7108

Closed eguiraud closed 1 year ago

eguiraud commented 3 years ago
vepadulano commented 3 years ago

I can't reproduce on my laptop (Fedora 32) with the following 4 config:

Python2 and ROOT release

vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: python2 --version
Python 2.7.18
vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: python2 reproducer_7108.py 
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/include/" -I"/usr/include/python2.7"
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/include/" -I"/usr/include/python2.7"

Python2 and ROOT debug

vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: python2 --version
Python 2.7.18
vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: python2 reproducer_7108.py 
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/include/" -I"/usr/include/python2.7"
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/include/" -I"/usr/include/python2.7"

Python3 and ROOT release

vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: python --version
Python 3.8.7
vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: python reproducer_7108.py 
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/include/" -I"/usr/include/python3.8"
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/include/" -I"/usr/include/python3.8"

Python3 and ROOT debug

vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: python --version
Python 3.8.7
vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: python reproducer_7108.py 
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/include/" -I"/usr/include/python3.8"
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/include/" -I"/usr/include/python3.8"
etejedor commented 3 years ago

I can reproduce with Enrico's reproducer, but not with:

import ROOT
print(ROOT.gSystem.GetIncludePath())

ROOT.gInterpreter.ProcessLine("""
auto v = std::vector<int>();
""")
print(ROOT.gSystem.GetIncludePath())

So it could be related to the use of the TCling interfaces.

etejedor commented 3 years ago

Instantiating the template is enough to reproduce (no need to create the object):

import ROOT
print(ROOT.gSystem.GetIncludePath())
ROOT.vector('int')
print(ROOT.gSystem.GetIncludePath())
etejedor commented 3 years ago

Using cppyy only has the extra path both before and after:

import cppyy
print(cppyy.gbl.gSystem.GetIncludePath())
cppyy.gbl.std.vector('int')
print(cppyy.gbl.gSystem.GetIncludePath())
eguiraud commented 3 years ago

I guess that means that the path is added by import cppyy. Does import ROOT not imply import cppyy?

etejedor commented 3 years ago

Yes, import ROOT imports cppyy too, so I don't think it's the cppyy import. And if you add import ROOT at the beginning of my last reproducer the first time we get the include paths is correct.

eguiraud commented 3 years ago

@vepadulano can you try with makefiles instead of ninja? I don't see any other difference

etejedor commented 3 years ago

More info: it's not a vector thing, it also happens if you instantiate e.g. a map or list or deque.

etejedor commented 3 years ago

This line in ROOT/__init__.py prevents cppyy from adding the path from the beginning.

`environ['CPPYY_API_PATH'] = 'none'`

but still does not explain why/where the instantiation adds the path.

eguiraud commented 3 years ago

the wrong include path is added here:

#0  0x00007fffefcba72e in cling::Interpreter::AddIncludePaths (this=0x5555559ac2f0, PathStr=..., Delm=0x0) at ../../../../../../../../root_dbg_includepaths/interpreter/cling/lib/Interpreter/Interpreter.cpp:626
#1  0x00007fffefcba926 in cling::Interpreter::AddIncludePath (this=0x5555559ac2f0, PathsStr=...) at ../../../../../../../../root_dbg_includepaths/interpreter/cling/lib/Interpreter/Interpreter.cpp:652
#2  0x00007fffefacc76d in TCling::AddIncludePath (this=0x555555958c90, path=0x7fffe5517ef8 "/home/eguiraud/ROOT/build_dbg_includepaths/include") at ../../../../root_dbg_includepaths/core/metacling/src/TCling.cxx:2644
#3  0x00007fffefac988e in TCling::RegisterModule (this=0x555555958c90, modulename=0x7fffe5517fce "libvectorDict", headers=0x7fffe552f190 <(anonymous namespace)::TriggerDictionaryInitialization_libvectorDict_Impl()::headers>, includePaths=0x7fffe552f1a0 <(anonymous namespace)::TriggerDictionaryInitialization_libvectorDict_Impl()::includePaths>, payloadCode=0x7fffe5517f30 "\n#line 1 \"libvectorDict dictionary payload\"\n\n\n#define _BACKWARD_BACKWARD_WARNING_H\n// Inline headers\n#include \"vector\"\n\n#undef  _BACKWARD_BACKWARD_WARNING_H\n", fwdDeclsCode=0x7fffe5517f2b "", triggerFunc=0x7fffe54fff24 <(anonymous namespace)::TriggerDictionaryInitialization_libvectorDict_Impl()>, fwdDeclsArgToSkip=std::vector of length 0, capacity 0, classesHeaders=0x7fffe55301d0 <(anonymous namespace)::TriggerDictionaryInitialization_libvectorDict_Impl()::classesHeaders>, lateRegistration=false, hasCxxModule=false) at ../../../../root_dbg_includepaths/core/metacling/src/TCling.cxx:2061
#4  0x00007ffff6f8797c in TROOT::RegisterModule (modulename=0x7fffe5517fce "libvectorDict", headers=0x7fffe552f190 <(anonymous namespace)::TriggerDictionaryInitialization_libvectorDict_Impl()::headers>, includePaths=0x7fffe552f1a0 <(anonymous namespace)::TriggerDictionaryInitialization_libvectorDict_Impl()::includePaths>, payloadCode=0x7fffe5517f30 "\n#line 1 \"libvectorDict dictionary payload\"\n\n\n#define _BACKWARD_BACKWARD_WARNING_H\n// Inline headers\n#include \"vector\"\n\n#undef  _BACKWARD_BACKWARD_WARNING_H\n", fwdDeclCode=0x7fffe5517f2b "", triggerFunc=0x7fffe54fff24 <(anonymous namespace)::TriggerDictionaryInitialization_libvectorDict_Impl()>, fwdDeclsArgToSkip=std::vector of length 0, capacity 0, classesHeaders=0x7fffe55301d0 <(anonymous namespace)::TriggerDictionaryInitialization_libvectorDict_Impl()::classesHeaders>, hasCxxModule=false) at ../../root_dbg_includepaths/core/base/src/TROOT.cxx:2561
#5  0x00007fffe54fff93 in (anonymous namespace)::TriggerDictionaryInitialization_libvectorDict_Impl () at G__vectorDict.cxx:982
#6  0x00007fffe54fffdf in (anonymous namespace)::DictInit::DictInit (this=0x7fffe55301d9 <(anonymous namespace)::__TheDictionaryInitializer>) at G__vectorDict.cxx:990
#7  0x00007fffe550011e in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at G__vectorDict.cxx:992
#8  0x00007fffe5500134 in _GLOBAL__sub_I_G__vectorDict.cxx(void) () at G__vectorDict.cxx:996
#9  0x00007ffff7fe437a in call_init (l=<optimized out>, argc=argc@entry=2, argv=argv@entry=0x7fffffffe4a8, env=env@entry=0x555556b4eb50) at dl-init.c:72
#10 0x00007ffff7fe4476 in call_init (env=0x555556b4eb50, argv=0x7fffffffe4a8, argc=2, l=<optimized out>) at dl-init.c:30
#11 _dl_init (main_map=0x555558799bf0, argc=2, argv=0x7fffffffe4a8, env=0x555556b4eb50) at dl-init.c:119
#12 0x00007ffff7fe82d3 in dl_open_worker (a=a@entry=0x7fffffffc6c0) at dl-open.c:517
#13 0x00007ffff7b4fb2f in __GI__dl_catch_exception (exception=<optimized out>, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:196
#14 0x00007ffff7fe7bba in _dl_open (file=0x555558d0abf0 "/home/eguiraud/ROOT/install_dbg_includepaths/lib/libvectorDict.so", mode=-2147483391, caller_dlopen=0x7fffefed6582 <cling::utils::platform::DLOpen(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*)+41>, nsid=<optimized out>, argc=2, argv=0x7fffffffe4a8, env=0x555556b4eb50) at dl-open.c:599
#15 0x00007ffff7f83256 in dlopen_doit (a=a@entry=0x7fffffffc8e0) at dlopen.c:66
#16 0x00007ffff7b4fb2f in __GI__dl_catch_exception (exception=exception@entry=0x7fffffffc880, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:196
#17 0x00007ffff7b4fbbf in __GI__dl_catch_error (objname=0x5555559ab150, errstring=0x5555559ab158, mallocedp=0x5555559ab148, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:215
#18 0x00007ffff7f83975 in _dlerror_run (operate=operate@entry=0x7ffff7f83200 <dlopen_doit>, args=args@entry=0x7fffffffc8e0) at dlerror.c:163
#19 0x00007ffff7f832e6 in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:87
#20 0x00007fffefed6582 in cling::utils::platform::DLOpen (Path="/home/eguiraud/ROOT/install_dbg_includepaths/lib/libvectorDict.so", Err=0x7fffffffc990) at ../../../../../../../../root_dbg_includepaths/interpreter/cling/lib/Utils/PlatformPosix.cpp:118
#21 0x00007fffefca7d0d in cling::DynamicLibraryManager::loadLibrary (this=0x555555a6e340, libStem="/home/eguiraud/ROOT/install_dbg_includepaths/lib/libvectorDict.so", permanent=true, resolved=false) at ../../../../../../../../root_dbg_includepaths/interpreter/cling/lib/Interpreter/DynamicLibraryManager.cpp:184
#22 0x00007fffefacfb91 in TCling::Load (this=0x555555958c90, filename=0x555558d06590 "/home/eguiraud/ROOT/install_dbg_includepaths/lib/libvectorDict.so", system=true) at ../../../../root_dbg_includepaths/core/metacling/src/TCling.cxx:3448
#23 0x00007ffff700d7c1 in TSystem::Load (this=0x55555597b4d0, module=0x555558d064f0 "/home/eguiraud/ROOT/install_dbg_includepaths/lib/libvectorDict.so", entry=0x0, system=true) at ../../../root_dbg_includepaths/core/base/src/TSystem.cxx:1940
#24 0x00007ffff711f8a1 in TUnixSystem::Load (this=0x55555597b4d0, module=0x555558d064f0 "/home/eguiraud/ROOT/install_dbg_includepaths/lib/libvectorDict.so", entry=0x0, system=true) at ../../../root_dbg_includepaths/core/unix/src/TUnixSystem.cxx:2789
#25 0x00007ffff6f86764 in TROOT::LoadClass (this=0x7ffff73a6bc0 <ROOT::Internal::GetROOT1()::alloc>, libname=0x555558cea6e0 "libvectorDict.so", check=false) at ../../root_dbg_includepaths/core/base/src/TROOT.cxx:2148
#26 0x00007fffefada682 in TCling::ShallowAutoLoadImpl (cls=0x555558cf04e0 "vector<bool>") at ../../../../root_dbg_includepaths/core/metacling/src/TCling.cxx:6052
#27 0x00007fffefada7c2 in TCling::DeepAutoLoadImpl (cls=0x555558cf04e0 "vector<bool>") at ../../../../root_dbg_includepaths/core/metacling/src/TCling.cxx:6075
#28 0x00007fffefadae02 in TCling::AutoLoad (this=0x555555958c90, cls=0x555558cf04e0 "vector<bool>", knowDictNotLoaded=true) at ../../../../root_dbg_includepaths/core/metacling/src/TCling.cxx:6191
#29 0x00007ffff70b1520 in TClass::LoadClassDefault (requestedname=0x555558cf04e0 "vector<bool>") at ../../../root_dbg_includepaths/core/meta/src/TClass.cxx:5763
#30 0x00007ffff70a7f0e in TClass::GetClass (name=0x555558cf3170 "std::vector<bool>", load=true, silent=true, hint_pair_offset=0, hint_pair_size=0) at ../../../root_dbg_includepaths/core/meta/src/TClass.cxx:3079
#31 0x00007ffff70a799d in TClass::GetClass (name=0x555558cf3170 "std::vector<bool>", load=true, silent=true) at ../../../root_dbg_includepaths/core/meta/src/TClass.cxx:2948
#32 0x00007ffff740a172 in Cppyy::GetScope (sname="std::vector<bool>") at ../../../../../root_dbg_includepaths/bindings/pyroot/cppyy/cppyy-backend/clingwrapper/src/clingwrapper.cxx:534
#33 0x00007ffff75602cb in CPyCppyy::Pythonize (pyclass=0x555558bf25d0, name="std::vector<int>") at ../../../../../root_dbg_includepaths/bindings/pyroot/cppyy/CPyCppyy/src/Pythonize.cxx:1106
#34 0x00007ffff7553f74 in CPyCppyy::CreateScopeProxy (name="std::vector<int>", parent=0x555556b95eb0) at ../../../../../root_dbg_includepaths/bindings/pyroot/cppyy/CPyCppyy/src/ProxyWrappers.cxx:686
#35 0x00007ffff7553aae in CPyCppyy::CreateScopeProxy (name="vector<int>", parent=0x0) at ../../../../../root_dbg_includepaths/bindings/pyroot/cppyy/CPyCppyy/src/ProxyWrappers.cxx:598
#36 0x00007ffff753146d in (anonymous namespace)::MakeCppTemplateClass (args=0x7fffeb472758) at ../../../../../root_dbg_includepaths/bindings/pyroot/cppyy/CPyCppyy/src/CPyCppyyModule.cxx:368
#37 0x000055555564bc73 in PyEval_EvalFrameEx ()
#38 0x0000555555642866 in PyEval_EvalCodeEx ()
#39 0x000055555565ff55 in ?? ()
#40 0x00005555556787a4 in ?? ()
#41 0x0000555555626883 in PyObject_Call ()
#42 0x00005555556da817 in ?? ()
#43 0x0000555555626883 in PyObject_Call ()
#44 0x000055555564a502 in PyEval_EvalFrameEx ()
#45 0x0000555555642866 in PyEval_EvalCodeEx ()
#46 0x00005555556421f9 in PyEval_EvalCode ()
#47 0x0000555555674e2f in ?? ()
#48 0x000055555566fd20 in PyRun_FileExFlags ()
#49 0x000055555566f6ca in PyRun_SimpleFileExFlags ()
#50 0x0000555555610188 in Py_Main ()
#51 0x00007ffff7a3f09b in __libc_start_main (main=0x55555560fbb0 <main>, argc=2, argv=0x7fffffffe4a8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe498) at ../csu/libc-start.c:308
#52 0x000055555560faea in _start ()
eguiraud commented 3 years ago

cling-only repro (PyROOT is off the hook :) ):

void foo() {
   std::cout << gInterpreter->GetIncludePath() << std::endl;
   gInterpreter->Load("/home/eguiraud/ROOT/install_dbg_includepaths/lib/libvectorDict.so");
   std::cout << gInterpreter->GetIncludePath() << std::endl;
}
eguiraud commented 3 years ago

Looks like it's a problem in the dictionary generation. The following snippet comes from G__vectorDict.cxx:

    static const char* includePaths[] = {                                                                               
"/home/eguiraud/ROOT/build_dbg_includepaths/include",                                                                   
"/home/eguiraud/ROOT/build_dbg_includepaths/include",                                                                   
0        
    };
vepadulano commented 3 years ago

@vepadulano can you try with makefiles instead of ninja? I don't see any other difference

Just finished compiling with cmake -GUnix\ Makefiles -Ddistcc=ON -Ddev=ON -DCMAKE_BUILD_TYPE=Debug -Dtesting=ON -Droottest=ON -DCMAKE_INSTALL_PREFIX=$ROOT_INSTALL/${PWD##*/} $ROOT_HOME and still get the same result

vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: source $ROOT_INSTALL/debugmakefiles/bin/thisroot.sh
vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: which root
~/Programs/rootproject/rootinstall/debugmakefiles/bin/root
vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: python reproducer_7108.py 
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/debugmakefiles/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/debugmakefiles/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/debugmakefiles/include/" -I"/usr/include/python3.8"
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/debugmakefiles/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/debugmakefiles/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/debugmakefiles/include/" -I"/usr/include/python3.8"
vepadulano commented 3 years ago

As for the includePaths variable in G__vectorDict.cxx of the build dir, here is what I get with the three different builds I've used so far

In rootbuild/debugmakefiles/core/clingutils/G__vectorDict.cxx

    static const char* includePaths[] = {
0

In rootbuild/devdebugtest/core/clingutils/G__vectorDict.cxx

    static const char* includePaths[] = {
0

In rootbuild/devrelease/core/clingutils/G__vectorDict.cxx

    static const char* includePaths[] = {
0
eguiraud commented 3 years ago

As far as I can tell those includes end up in the dictionary because they are passed to TModuleGenerator::ParseArgs from RootClingMain at rootcling_impl.cxx:4491 (modGen.ParseArgs(pcmArgs)), which saves them in TModuleGenerator::fCompI and later writes them to the dictionary when TModuleGenerator::WriteRegistrationSourceImpl is called.

RootClingMain adds those includes to pcmArgs at rootcling_impl.cxx:4097 and following lines. They originally come from clingArgs, which gets them from the commandline arguments.

I don't see a way this could not happen, so figuring out what's different in Vincenzo's case should be interesting.

eguiraud commented 3 years ago

@vgvassilev @Axel-Naumann ping!

vgvassilev commented 3 years ago

It may make sense to register such include paths as "private" and adjust the interface of gSystem->GetIncludePath (cc:pcanal).

In principle we do not need this injected include path when runtime_cxxmodules are on and we might just drop that part from dictionary generation. There might be some code out there that was relying on this gray area include path.

I'd defer that to @Axel-Naumann.

eguiraud commented 3 years ago

We've been trying to figure out why Vincenzo doesn't see this. It turns it's regexes string matching: his build path starts with his source path, and that makes rootcling_impl drop buildpath/include from the list of header paths that are written to the dictionary. Offending logic is here:

https://github.com/root-project/root/blob/730d24b73296a7d0b78d33588a5c9fdbb80e6731/core/dictgen/src/rootcling_impl.cxx#L4097-L4115

We think the fix should be put in ROOT_GENERATE_DICTIONARY, that already sets some excludePaths e.g. at https://github.com/root-project/root/blob/68ffb0bda479cd1fd10908f1264e8d42c29339f6/cmake/modules/RootMacros.cmake#L329

and https://github.com/root-project/root/blob/68ffb0bda479cd1fd10908f1264e8d42c29339f6/cmake/modules/RootMacros.cmake#L463

bendavid commented 3 years ago

Adding ${CMAKE_BINARY_DIR}/include to line 329 fixes the problem we were having with this when building root into a singularity image at least.

eguiraud commented 3 years ago

Possibly an instance of this problem on the forum: https://root-forum.cern.ch/t/compilation-failing-because-root-is-searching-for-files-in-the-source-directory-and-not-in-the-installation-directory/42686

bendavid commented 3 years ago

Some insight into when/where this actually causes a problem.

Take for example a recent CMSSW release where we indeed end up with a bunch of stuff in the include path starting with "/data/cmsbld/jenkins/...". If I try to do anything with gSystem->CompileMacro then things are fine if /data/cmsbld doesn't exist on the machine where I'm running, but if I create that directory and set permissions to make it inaccessible then I get a permission denied error when compiling a macro.

This also explains why I was having problems with this specifically in my singularity images, because in this case the build path was in /root/root_build, where /root still exists but is inaccessible by normal users in the final singularity image.

vgvassilev commented 2 years ago

Looks like it's a problem in the dictionary generation. The following snippet comes from G__vectorDict.cxx:

    static const char* includePaths[] = {                                                                               
"/home/eguiraud/ROOT/build_dbg_includepaths/include",                                                                   
"/home/eguiraud/ROOT/build_dbg_includepaths/include",                                                                   
0        
    };

Libraries are free to use the interpreter during their static initialization after being loaded. In that case this library decided to add an extra include path. If we do not need this include path (eg there is no useful header to resolve from there). In fact we ignore such include paths here. What is build_dbg_includepaths? Is it your build folder?

eguiraud commented 2 years ago

Hi @vgvassilev , I don't have the repro set up anymore, sorry

Ah but yes, build_dbg_includepaths was my build directory, that's the problem.

As Josh mentions above adding ${CMAKE_BINARY_DIR}/include to the excluded paths at the code location you mention fixes the problem.

Axel-Naumann commented 2 years ago

@vgvassilev

In principle we do not need this injected include path when runtime_cxxmodules are on and we might just drop that part from dictionary generation.

Isn't the recent nlohmann_json issue contradicting this? These headers must be found at runtime, and that's happening either because they are found where they were during build time, or due to ROOT_INCLUDE_PATH. Unless "this" in "we do not need this injected include path" specifically refers to the ROOT build directory. I suppose we'd "survive" that because we anyway inject $ROOTSYS/include (with ROOTSYS really being "install include dir or parent dir of location of libCore") at runtime.

vgvassilev commented 2 years ago

@Axel-Naumann, yeah, "this" means "ROOT build directory". I suspect that we did not capture properly which is the build directory. I remember some discussion about this with @linev where he reported he could not drop the root_build/include iirc.

linev commented 2 years ago

I do not remember all details, but one reason why root_build/include may be still required - builtins libraries. Includes of that builtins installed in root_build/include directory and must be there when ROOT is running.

For instance - root_build/include/vdt directory which is used by RVec.hxx. Or root_build/include/VecCore, which are used by Math/types.h

linev commented 2 years ago

But maybe one can resolve this - root_build/include as include path required only during ROOT compilation. But should not be inserted into generated ROOT dictionaries - $ROOTSYS/include anyway accessible during ROOT running. No idea how this can be done.

eguiraud commented 2 years ago

Ping :) it would be nice to fix this for v6.26

Axel-Naumann commented 2 years ago

Should be fixed in master and v6-26-patches. Can you confirm?

Is this also needed in 6.24?

bendavid commented 2 years ago

this doesn't fix it for me unfortunately

root -l testtref.cpp+
root [0] 
Processing testtref.cpp+...
Info in <TUnixSystem::ACLiC>: creating shared library /home/b/bendavid/wmassdev/./testtref_cpp.so
cc1plus: error: /var/aurbuild/root-git/src/build/include: Permission denied
cc1plus: error: /var/aurbuild/root-git/src/build/include/: Permission denied
/usr/sbin/ld: cannot find /home/b/bendavid/wmassdev/testtref_cpp_ACLiC_dict.o: No such file or directory
collect2: error: ld returned 1 exit status
Error in <ACLiC>: Executing 'cd "/home/b/bendavid/wmassdev" ; c++ -fPIC -c -O3 -DNDEBUG -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS -DIS_RPATH_BUILD=1 -pthread -g -fvar-tracking-assignments -fdebug-prefix-map=/var/aurbuild/root-git/src=/usr/src/debug -std=c++17 -Wno-implicit-fallthrough -Wno-noexcept-type -pipe -W -Woverloaded-virtual -fsigned-char -pthread  -I/usr/include -I"/etc/root" -I"/etc/root/cling" -I"/etc/root/cling/plugins/include" -I"/usr/include" -I"/var/aurbuild/root-git/src/build/include" -I"/var/aurbuild/root-git/src/build/include/"   -D__ACLIC__ "/home/b/bendavid/wmassdev/testtref_cpp_ACLiC_dict.cxx" ; c++ -O3 -DNDEBUG "/home/b/bendavid/wmassdev/testtref_cpp_ACLiC_dict.o" -shared   "/usr/lib/libm.so" "/usr/lib/libdl.so" "/usr/lib/libc.so" "/usr/lib/libgcc_s.so" "/usr/lib/libstdc++.so" "/usr/lib/root/libRint.so" "/usr/lib/root/libCore.so" "/usr/lib/libpcre.so.1" "/usr/lib/liblzma.so.5" "/usr/lib/libxxhash.so.0" "/usr/lib/liblz4.so.1" "/usr/lib/libz.so.1" "/usr/lib/libzstd.so.1" "/usr/lib/libpthread.so.0" "/usr/lib/libnss_files.so.2" "/usr/lib/root/libRIO.so" "/usr/lib/root/libThread.so" "/usr/lib/libtbb.so.12" "/usr/lib/root/libCling.so" "/usr/lib/libncursesw.so.6" "/usr/lib/root/libMathCore.so" "/usr/lib/root/libImt.so" "/usr/lib/root/libMultiProc.so" "/usr/lib/root/libNet.so" "/usr/lib/libssl.so.1.1" "/usr/lib/libcrypto.so.1.1" "/usr/lib/libnss_systemd.so.2" "/usr/lib/librt.so.1" "/usr/lib/libcrypt.so.2" "/usr/lib/libp11-kit.so.0" "/usr/lib/libffi.so.8" -o "/home/b/bendavid/wmassdev/./testtref_cpp.so"' failed!
eguiraud commented 2 years ago

Uh, disappointing. My repro at the top of the issue seems to be fixed:

/tmp python
Python 3.10.1 (main, Dec 18 2021, 23:53:45) [GCC 11.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ROOT
>>> print(ROOT.gSystem.GetIncludePath())
-I$ROOTSYS/include -I"/home/blue/ROOT/master/install/etc/" -I"/home/blue/ROOT/master/install/etc//cling" -I"/home/blue/ROOT/master/install/etc//cling/plugins/include" -I"/home/blue/ROOT/master/install/include/" -I"/usr/include/python3.10"
>>> ROOT.vector('int')()
<cppyy.gbl.std.vector<int> object at 0x55a00ceb0a00>
>>> print(ROOT.gSystem.GetIncludePath())
-I$ROOTSYS/include -I"/home/blue/ROOT/master/install/etc/" -I"/home/blue/ROOT/master/install/etc//cling" -I"/home/blue/ROOT/master/install/etc//cling/plugins/include" -I"/home/blue/ROOT/master/install/include/" -I"/usr/include/python3.10"

(no build directory in the include paths anymore)

bendavid commented 2 years ago

I think you can easily reproduce my error by keeping the build directory present but making it inaccessible by permissions.

eguiraud commented 2 years ago

I can't, sorry. I did the following:

As root user:

$ chown -R root:root cmake-build-dir
$ chmod -R o-rwx cmake-build-dir

As my user:

$ cd cmake-build-dir
cd: Permission denied: “cmake-build-dir”
$ ls cmake-build-dir
ls: cannot open directory 'cmake-build-dir': Permission denied

And still:

$ source ~/ROOT/master/install/bin/thisroot.fish
$ cat foo.cpp
#include <memory>
#include <ROOT/RDataFrame.hxx>
#include <vector>

void foo() {
      ROOT::RDataFrame df(10);
      std::vector<int> v;
}

$ root -l foo.cpp+
root [0]
Processing foo.cpp+...
Info in <TUnixSystem::ACLiC>: creating shared library /tmp/./foo_cpp.so
root [1]
bendavid commented 2 years ago

I also can't reproduce this in a standalone build, only happens when I install using the system package manager (in this case archlinux in a podman image). Maybe it has to do with $ROOTSYS being set vs not.

Let me see if I can provoke it in a more standalone manner.

On the other hand if I grep for the build directory, it does still appear in the final .so and .pcm files.

eguiraud commented 2 years ago

On the other hand if I grep for the build directory, it does still appear in the final .so and .pcm files.

That might or might not be a problem, I guess?

With the original reproducer fixed I'm not sure how to make progress here :/

eguiraud commented 1 year ago

Closing since the original reproducer has been fixed.