search

grasph / wrapit

Automatization of C++--Julia wrapper generation
MIT License
98 stars 12 forks source link

Type `int` in the generated code for function arguments of a different type. #30

Closed bcicc closed 7 months ago

bcicc commented 10 months ago

After setting up Ex002 per instructions and running make all, there is a failure. I did not modify the Makefile. Below is the complete terminal output leading up to the failure:

make check_root
make[1]: Entering directory '/home/ben/dev/wrapperTest/ex002-ROOT'
make[1]: Nothing to be done for 'check_root'.
make[1]: Leaving directory '/home/ben/dev/wrapperTest/ex002-ROOT'
./wrapit --force ROOT-generated.wit
/home/ben/miniconda3/envs/rootEnv/include/RtypesCore.h:28:10: 'cstddef' file not found
Warning: failed to determine if some argument of function printValue has a default value. This can happen if the function is defined  with the help of a macro. Support of default argument value disabled for this function.
Warning: failed to determine if some argument of function GetHomeDirectory has a default value. This can happen if the function is defined  with the help of a macro. Support of default argument value disabled for this function.
Warning: failed to determine if some argument of function AddExtraInterpreterArgs has a default value. This can happen if the function is defined  with the help of a macro. Support of default argument value disabled for this function.
Warning: failed to determine if some argument of function printValue has a default value. This can happen if the function is defined  with the help of a macro. Support of default argument value disabled for this function.
Warning: method Build found at /home/ben/miniconda3/envs/rootEnv/include/TF1.h:735:32 skipped because the definition of its class was not found.
Warning: no wrapper will be produced for function 'Info(const char *, const char *, ...)  const' because of lack of support for variadic functions.
Warning: no wrapper will be produced for function 'Warning(const char *, const char *, ...)  const' because of lack of support for variadic functions.
Warning: no wrapper will be produced for function 'Error(const char *, const char *, ...)  const' because of lack of support for variadic functions.
Warning: no wrapper will be produced for function 'SysError(const char *, const char *, ...)  const' because of lack of support for variadic functions.
Warning: no wrapper will be produced for function 'Fatal(const char *, const char *, ...)  const' because of lack of support for variadic functions.

Generated wrapper statictics
  enums:            38
  classes/structs:  59
    templates:      1
    others:         58
  class methods:    1557
  field accessors:  47 getters and 47 setters
  global variable accessors: 10 getters and 7 setters
  global functions: 58

/home/ben/miniconda3/envs/rootEnv/bin/x86_64-conda-linux-gnu-c++ -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/ben/miniconda3/envs/rootEnv/include -std=c++17 -I'/home/ben/.julia/juliaup/julia-1.9.3+0.x64.linux.gnu/include/julia' -fPIC -fmax-errors=3 -O2 -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/ben/miniconda3/envs/rootEnv/include -MMD -I /home/ben/miniconda3/envs/rootEnv/include  -c -I /home/ben/.julia/artifacts/88a033de19250acca6784647964d43d7121a06aa/include -std=c++17 -o jlROOT.o jlROOT.cxx
jlROOT.cxx: In function 'void define_julia_module(jlcxx::Module&)':
jlROOT.cxx:1084:30: error: invalid 'static_cast' from type 'void (TObject::*)(std::ostream&, Option_t*)' {aka 'void (TObject::*)(std::basic_ostream<char>&, const char*)'} to type 'void (TObject::*)(int&, Option_t*)' {aka 'void (TObject::*)(int&, const char*)'}
 1084 |   t0.method("SavePrimitive", static_cast<void (TObject::*)(int &, Option_t *) >(&TObject::SavePrimitive));
      |                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
jlROOT.cxx: In lambda function:
jlROOT.cxx:1085:80: error: cannot convert 'int' to 'std::ostream&' {aka 'std::basic_ostream<char>&'}
 1085 |   t0.method("SavePrimitive", [](TObject& a, int & arg0)->void{ a.SavePrimitive(arg0); });
      |                                                                                ^~~~
      |                                                                                |
      |                                                                                int
In file included from /home/ben/miniconda3/envs/rootEnv/include/TMatrixTBase.h:69,
                 from /home/ben/miniconda3/envs/rootEnv/include/TMatrixT.h:23,
                 from /home/ben/miniconda3/envs/rootEnv/include/TVectorT.h:23,
                 from jlROOT.h:1,
                 from jlROOT.cxx:7:
/home/ben/miniconda3/envs/rootEnv/include/TObject.h:163:52: note:   initializing argument 1 of 'virtual void TObject::SavePrimitive(std::ostream&, Option_t*)'
  163 |    virtual void        SavePrimitive(std::ostream &out, Option_t *option = "");
      |                                      ~~~~~~~~~~~~~~^~~
jlROOT.cxx: In lambda function:
jlROOT.cxx:1086:81: error: cannot convert 'int' to 'std::ostream&' {aka 'std::basic_ostream<char>&'}
 1086 |   t0.method("SavePrimitive", [](TObject* a, int & arg0)->void{ a->SavePrimitive(arg0); });
      |                                                                                 ^~~~
      |                                                                                 |
      |                                                                                 int
/home/ben/miniconda3/envs/rootEnv/include/TObject.h:163:52: note:   initializing argument 1 of 'virtual void TObject::SavePrimitive(std::ostream&, Option_t*)'
  163 |    virtual void        SavePrimitive(std::ostream &out, Option_t *option = "");
      |                                      ~~~~~~~~~~~~~~^~~
compilation terminated due to -fmax-errors=3.
make: *** [Makefile:60: jlROOT.o] Error 1
grasph commented 10 months ago

It looks like an issue with your compiler installation as it does not find the standard library cstddef. You can check it doing the following test.

cat <<EOF > test.cc
#include <cstddef>
EOF

g++ -c test.cc

If the compilation fails, it will confirm the issue comes your g++ compiler installation.

Philippe.

bcicc commented 10 months ago

That compilation test succeeded. Is the issue maybe because root is in a conda env?

bcicc commented 10 months ago

Should I be using Clang-11 or a newer version? The readme.md says clang version 11, but not sure if that's just for building wrapit.

biona001 commented 8 months ago

I'm having a similar issue, except I'm failing at Ex001. 

$ wrapit Hello.wit --force
/Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk/usr/include/wchar.h:89:10: 'stdarg.h' file not found

Generated wrapper statictics
  enums:            0
  classes/structs:  1
    templates:      0
    others:         1
  class methods:    2
  field accessors:  0 getters and 0 setters
  global variable accessors: 0 getters and 0 setters
  global functions: 0

Any insights/tips/suggestions would be highly appreciated

grasph commented 8 months ago

You can try adding in the wrapit configuration file:

clang_opts      = [  "-resource-dir", "DIR" ]

with DIR replaced by the clang resource directory path you will find by running from your shell the command:

clang -print-resource-dir

Philippe.

grasph commented 7 months ago

With commit 72ae291 5ad6702 explicit setting of resource-dir should not be needed anymore (although tested on Linux but not Max OS X).

@biona001, @termi-official, if you can try, please confirm you don't need to set resource-dir anymore.

biona001 commented 7 months ago

On mac, I'm still getting the same error:

$ git clone https://github.com/grasph/wrapit
$ cd wrapit/examples/ex001-HelloWorld/
$ wrapit Hello.wit --force
/Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk/usr/include/wchar.h:89:10: 'stdarg.h' file not found

Generated wrapper statictics
  enums:            0
  classes/structs:  1
    templates:      0
    others:         1
  class methods:    2
  field accessors:  0 getters and 0 setters
  global variable accessors: 0 getters and 0 setters
  global functions: 0

Setting resource-dir fixes the problem as before.

I didn't test on linux.

grasph commented 7 months ago

Thanks for the report. Could you run with -v 1 option, which will display the resource directory selected automatically, and compare it with the one you use when you set it in the configuration file?

The first commit I sent had a problem, can you confirm you use 5ad6702 ?

termi-official commented 7 months ago

Also fails for me unfortunately

julia> include("runtests.jl")
make: Entering directory '/home/dogiermann/Repos/wrapit/test/TestSizet'
rm -f libTestSizet/libjlTestSizet.so libTestSizet/src/jlTestSizet.cxx libTestSizet/src/jlTestSizet.h TestSizet/src/TestSizet.jl -r libTestSizet/build jlTestSizet-report.txt libTestSizet/build/jlTestSizet.o  libTestSizet/build/jlTestSizet.d
[ -d libTestSizet/src ] && cd libTestSizet/src && rm -f  dbg_msg.h  generated_cxx  Wrapper.h || true
rmdir -p libTestSizet/src TestSizet/src 2>/dev/null || true
make all && . ./setup.sh && julia runTestSizet.jl
make[1]: Entering directory '/home/dogiermann/Repos/wrapit/test/TestSizet'
/home/dogiermann/Repos/wrapit/build/wrapit -v 1 --force TestSizet.wit
Clang resource directory: /home/dogiermann/Repos/wrapit/lib/clang/13.0.1
WARNING: file '/home/dogiermann/Repos/wrapit/lib/clang/13.0.1/include/stddef.h' was not found. Please check the clang resource directory is set properly: see clang manual and use wrapit clang_opts parameter to pass options to clang.
Clang options:
    -x
    c++-header
    -std=c++17
    -I
    .
    -I
    /home/opt/llvm-dev/include/c++/v1/
    -resource-dir
    /home/dogiermann/Repos/wrapit/lib/clang/13.0.1

/usr/lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cstddef:50:10: 'stddef.h' file not found

Generated wrapper statictics
  enums:            0
  classes/structs:  0
    templates:      0
    others:         0
  class methods:    0
  field accessors:  0 getters and 0 setters
  global variable accessors: 0 getters and 0 setters
  global functions: 1

[ -d libTestSizet/build ] || mkdir libTestSizet/build
/home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/bin/clang++ -I'/home/dogiermann/Tools/julia-1.9.4/include/julia' -fPIC -Wno-unused-variable -Wno-unused-but-set-variable -fmax-errors=3 -I. -MMD  -c -I /home/dogiermann/.julia/artifacts/8dbf3a116981de15f84137eda9d8c478f57d07a9/include -std=c++20 -o libTestSizet/build/jlTestSizet.o libTestSizet/src/jlTestSizet.cxx
clang-13: warning: argument unused during compilation: '-fmax-errors=3' [-Wunused-command-line-argument]
libTestSizet/src/jlTestSizet.cxx:20:19: error: static_cast from 'size_t (*)()' (aka 'unsigned long (*)()') to 'int (*)()' is not allowed
    t.method("f", static_cast<int (*)() >(&f));
                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make[1]: *** [../make.rules:40: libTestSizet/build/jlTestSizet.o] Error 1
make[1]: Leaving directory '/home/dogiermann/Repos/wrapit/test/TestSizet'
make: *** [../make.rules:50: test] Error 2
make: Leaving directory '/home/dogiermann/Repos/wrapit/test/TestSizet'
TestSizet: Test Failed at /home/dogiermann/Repos/wrapit/test/runtests.jl:11
  Expression: begin
    try
        run(`make -C $t clean test`)
        true
    catch
        false
    end
end

Stacktrace:
 [1] macro expansion
   @ ~/Tools/julia-1.9.4/share/julia/stdlib/v1.9/Test/src/Test.jl:478 [inlined]
 [2] macro expansion
   @ ~/Repos/wrapit/test/runtests.jl:11 [inlined]
 [3] macro expansion
   @ ~/Tools/julia-1.9.4/share/julia/stdlib/v1.9/Test/src/Test.jl:1498 [inlined]
 [4] macro expansion
   @ ~/Repos/wrapit/test/runtests.jl:11 [inlined]
 [5] macro expansion
   @ ~/Tools/julia-1.9.4/share/julia/stdlib/v1.9/Test/src/Test.jl:1498 [inlined]
 [6] top-level scope
   @ ~/Repos/wrapit/test/runtests.jl:9
grasph commented 7 months ago

Due to a bug in 5ad6702 clang resource directory was set only if verbosity level was set to 1 or above. Commit 176af5a fixes this and various issues with MacOS.

@termi-official, is /home/dogiermann/Repos/wrapit/lib/clang/13.0.1 the path you provide as resource directory when you set it explicitly ? The wrapit logs report that the file /home/dogiermann/Repos/wrapit/lib/clang/13.0.1/include/stddef.h is missing, while this file is included in the tarball released by the LLVM team for darwin, https://github.com/llvm/llvm-project/releases/download/llvmorg-16.0.3/clang+llvm-16.0.3-arm64-apple-darwin22.0.tar.xz . With -v 2 the clang search directory list for header file is displayed in the logs. One of these directories must contain the stddef.h file for the Sizet test to work.

termi-official commented 7 months ago

Thanks for working on this. No, I have not passed any resource directly explicitly here. I just executed the tests of wrapit locally. Explicitly setting the resource directory works, tho.

> clang -print-resource-dir                                                                                                                                                                                                         
/home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/clang/13.0.1

I can still see the same failure on https://github.com/grasph/wrapit/commit/176af5a89d7da56630253915cf16b226ee4aee14 . I think the issue here is might be related to using spack (which I have to load into a shell to function correctly) instead of using the system's LLVM. However, the latter is no option because on arch based system the clang tooling is not provided (IIUC).

Maybe it makes sense to just error early with a printout of the current resource directory and the option that it might need to be set manually in some cases?

grasph commented 7 months ago

Hello @termi-official . Having to set the resource directory by hand in special cases it's indeed fine. Nevertheless, I don't understand why the automatic setting does not work in your case. I've tried to install llvm+clang with spack to reproduce the issue, but I have some issue with spack preventing the installation. Could you do the following:

Philippe.

termi-official commented 7 months ago

First the output after spack load llvm@13

> which clang
/home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/bin/clang
> ldd `which clang` 
    linux-vdso.so.1 (0x00007ffff97ec000)
    libz.so.1 => /usr/lib/libz.so.1 (0x00007fd0af92a000)
    libncursesw.so.6 => /usr/lib/libncursesw.so.6 (0x00007fd0af8b3000)
    libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fd0a8400000)
    libm.so.6 => /usr/lib/libm.so.6 (0x00007fd0a8713000)
    libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007fd0af88c000)
    libc.so.6 => /usr/lib/libc.so.6 (0x00007fd0a821e000)
    /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fd0af984000)
> which wrapit
wrapit: aliased to /home/dogiermann/Repos/wrapit/build/wrapit
> ldd wrapit
    linux-vdso.so.1 (0x00007ffdd38cd000)
    libclang.so.13 => /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libclang.so.13 (0x00007fa2ff000000)
    libm.so.6 => /usr/lib/libm.so.6 (0x00007fa2fef13000)
    libz.so.1 => /usr/lib/libz.so.1 (0x00007fa303089000)
    libncursesw.so.6 => /usr/lib/libncursesw.so.6 (0x00007fa303012000)
    libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fa2fec00000)
    libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007fa2feeee000)
    libc.so.6 => /usr/lib/libc.so.6 (0x00007fa2fea1e000)
    /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fa3030e3000)

The path /home/dogiermann/Repos/wrapit/bin/ does not exist on my system.

For a fresh shell I get

> which clang
/usr/bin/clang
> ldd `which clang`
    linux-vdso.so.1 (0x00007ffc662de000)
    libclang-cpp.so.16 => /usr/lib/libclang-cpp.so.16 (0x00007feee8200000)
    libLLVM-16.so => /usr/lib/libLLVM-16.so (0x00007feedfe00000)
    libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007feedfa00000)
    libc.so.6 => /usr/lib/libc.so.6 (0x00007feedf81e000)
    libm.so.6 => /usr/lib/libm.so.6 (0x00007feeeb76d000)
    libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007feeeb748000)
    /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007feeeb8c0000)
    libffi.so.8 => /usr/lib/libffi.so.8 (0x00007feeeb73b000)
    libedit.so.0 => /usr/lib/libedit.so.0 (0x00007feeeb6ff000)
    libz.so.1 => /usr/lib/libz.so.1 (0x00007feeeb6e5000)
    libzstd.so.1 => /usr/lib/libzstd.so.1 (0x00007feedfd2d000)
    libncursesw.so.6 => /usr/lib/libncursesw.so.6 (0x00007feedfcb6000)
    libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007feedf6b8000)
    liblzma.so.5 => /usr/lib/liblzma.so.5 (0x00007feeeb6b0000)
    libicuuc.so.73 => /usr/lib/libicuuc.so.73 (0x00007feedf400000)
    libicudata.so.73 => /usr/lib/libicudata.so.73 (0x00007feedd400000)
grasph commented 7 months ago

Thanks @termi-official . Are you sure you were in the same conditions when you made the test reported in this comment?

Indeed, I would expect the directory discovery to work in these conditions. Indeed, the path is built from the path of thelibclang.so library wrapit is dynamically linked to. From the ldd `which wrapit` output the latter is,

/home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libclang.so.13

This should lead to the correct resource directory path, /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/clang/13.0.1.

Philippe.

termi-official commented 7 months ago

Thanks @termi-official . Are you sure you were in the same conditions when you made the test reported in this comment?

Yes. I tried again on 176af5a89d and can still reproduce. Here my full command line

    ~/Repos/wrapit/build    main !4 ?30  . /home/dogiermann/Repos/spack/share/spack/setup-env.sh                                                                                                                                                                           INT ✘                                                                                                                                                                                                                                                                               ~/Repos/wrapit/build    main !4 ?30  spack load llvm@13.0.1                                                                                                                                                                                                                ✔ 
    ~/Repos/wrapit/build    main !4 ?30  make clean                                                                                                                                                                                                                            ✔ 
    ~/Repos/wrapit/build    main !4 ?30  make -j8                                                                                                                                                                                                                              ✔ 
[  0%] Built target version
[ 11%] Building CXX object CMakeFiles/wrapit.dir/src/CodeTree.cpp.o
[ 22%] Building CXX object CMakeFiles/wrapit.dir/src/TypeRcd.cpp.o
[ 33%] Building CXX object CMakeFiles/wrapit.dir/src/libclang-ext.cpp.o
[ 44%] Building CXX object CMakeFiles/wrapit.dir/src/utils.cpp.o
[ 55%] Building CXX object CMakeFiles/wrapit.dir/src/FunctionWrapper.cpp.o
[ 66%] Building CXX object CMakeFiles/wrapit.dir/src/main.cpp.o
[ 77%] Building CXX object CMakeFiles/wrapit.dir/version.cpp.o
[ 88%] Linking CXX executable wrapit
[100%] Built target wrapit
    ~/Repos/wrapit/build    main !4 ?30  alias wrapit=$HOME/Repos/wrapit/build/wrapit                                                                                                                                                                                  ✔  12s  
    ~/Repos/wrapit/build    main !4 ?30  cd ../test                                                                                                                                                                                                                            ✔ 
    ~/Repos/wrapit/test    main !4 ?30  julia                                                                                                                                                                                                                                  ✔ 
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.4 (2023-11-14)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> include("runtests.jl")
make: Entering directory '/home/dogiermann/Repos/wrapit/test/TestSizet'
rm -f libTestSizet/libjlTestSizet.so libTestSizet/src/jlTestSizet.cxx libTestSizet/src/jlTestSizet.h TestSizet/src/TestSizet.jl -r libTestSizet/build jlTestSizet-report.txt libTestSizet/build/jlTestSizet.o  libTestSizet/build/jlTestSizet.d
[ -d libTestSizet/src ] && cd libTestSizet/src && rm -f jlTestSizet.cxx jlTestSizet.cxx dbg_msg.h  generated_cxx  Wrapper.h || true
rmdir -p libTestSizet/src TestSizet/src 2>/dev/null || true
make all && . ./setup.sh && julia runTestSizet.jl
make[1]: Entering directory '/home/dogiermann/Repos/wrapit/test/TestSizet'
/home/dogiermann/Repos/wrapit/build/wrapit -v 1 --force TestSizet.wit
Clang resource directory: /home/dogiermann/Repos/wrapit/lib/clang/13.0.1
WARNING: file '/home/dogiermann/Repos/wrapit/lib/clang/13.0.1/include/stddef.h' was not found. Please check the clang resource directory is set properly: see clang manual and use wrapit clang_opts parameter to pass options to clang.
Clang options:
    -x
    c++-header
    -std=c++17
    -I
    .
    -I
    /home/opt/llvm-dev/include/c++/v1/
    -resource-dir
    /home/dogiermann/Repos/wrapit/lib/clang/13.0.1

/usr/lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cstddef:50:10: 'stddef.h' file not found

Indeed, I would expect the directory discovery to work in these conditions. Indeed, the path is built from the path of thelibclang.so library wrapit is dynamically linked to. From the ldd `which wrapit` output the latter is,

The failure is here https://github.com/grasph/wrapit/blob/0126efb1fb9852fc6d6bbd784d4149274c640afc/src/utils.cpp#L114-L122 . dli_fname contains the path in which I have built wrapit and not the clang binary path. Checking this manually I can confirm that clang_getCursorType is in my wrapit executable due to static linking - turns out I only have the static libraries for tooling (which is the default for the spack llvm package).

termi-official commented 7 months ago

I think if the path should be fixed like this, then it is just possible to grab the resource path in cmake, generate a file with the path as a constant, and access this constant during runtime.

grasph commented 7 months ago

I see. I hadn't considered the case of a static link.

From the ldd `which wrapit` output it looks like it is dynamically linked to /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libclang.so.13.

Could it be that it is linked twice, statically and dynamically?

It's standard to have the LibTooling libraries, apart from libclang, statically linked. clang_getCursor comes from libclang library, which the code is assuming to be dynamically linked.

termi-official commented 7 months ago

From the ldd `which wrapit` output it looks like it is dynamically linked to /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libclang.so.13.

Could it be that it is linked twice, statically and dynamically?

It's standard to have the LibTooling libraries, apart from libclang, statically linked. clang_getCursor comes from libclang library, which the code is assuming to be dynamically linked.

Good point. Doesn't seem to be the case tho that it gets statically linked, which wonders me. Running the make command verbose gives me the following information for the linking step:

/home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/bin/clang++ -rdynamic CMakeFiles/wrapit.dir/src/TypeRcd.cpp.o CMakeFiles/wrapit.dir/src/utils.cpp.o "CMakeFiles/wrapit.dir/src/libclang-ext.cpp.o" CMakeFiles/wrapit.dir/src/FunctionWrapper.cpp.o CMakeFiles/wrapit.dir/src/CodeTree.cpp.o CMakeFiles/wrapit.dir/src/main.cpp.o CMakeFiles/wrapit.dir/version.cpp.o -o wrapit  /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libclang.so.13.0.1 /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libclangAST.a /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libclangDriver.a -ldl /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libclangLex.a /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libLLVMFrontendOpenMP.a /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libLLVMTransformUtils.a /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libLLVMAnalysis.a /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libLLVMObject.a /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libLLVMBitReader.a /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libLLVMMCParser.a /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libLLVMMC.a /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libLLVMDebugInfoCodeView.a /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libLLVMTextAPI.a /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libclangBasic.a /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libLLVMOption.a /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libLLVMProfileData.a /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libLLVMCore.a /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libLLVMBinaryFormat.a /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libLLVMRemarks.a /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libLLVMBitstreamReader.a /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libLLVMSupport.a -lrt -ldl -lm /usr/lib/libz.so -ltinfo /home/dogiermann/Repos/spack/opt/spack/linux-manjaro21-zen/gcc-11.2.0/llvm-13.0.1-ujjeg2kym5yi5vzurktjtknit5xexaow/lib/libLLVMDemangle.a

where I don't see anything obviously wrong.

My local patch for the resource directory is

std::string get_resource_dir(){
  std::string p = get_so_path("libclang.dll", reinterpret_cast<void*>(clang_getCursorType));
  if(p.size() == 0){
    std::cout << "get_resource_dir 1:" << p << std::endl;
    return p;
  } else{
    std::string custom_resource_dir("");
    auto dir = clang::driver::Driver::GetResourcesPath(p, custom_resource_dir);
    std::cout << "get_resource_dir 2:" << p << " | " << dir << std::endl;
    return dir;
  }
}

and the corresponding output is

julia> include("runtests.jl")
make: Entering directory '/home/dogiermann/Repos/wrapit/test/TestSizet'
rm -f libTestSizet/libjlTestSizet.so libTestSizet/src/jlTestSizet.cxx libTestSizet/src/jlTestSizet.h TestSizet/src/TestSizet.jl -r libTestSizet/build jlTestSizet-report.txt libTestSizet/build/jlTestSizet.o  libTestSizet/build/jlTestSizet.d
[ -d libTestSizet/src ] && cd libTestSizet/src && rm -f jlTestSizet.cxx jlTestSizet.cxx dbg_msg.h  generated_cxx  Wrapper.h || true
rmdir -p libTestSizet/src TestSizet/src 2>/dev/null || true
make all && . ./setup.sh && julia runTestSizet.jl
make[1]: Entering directory '/home/dogiermann/Repos/wrapit/test/TestSizet'
/home/dogiermann/Repos/wrapit/build/wrapit -v 1 --force TestSizet.wit
get_resource_dir 2:/home/dogiermann/Repos/wrapit/build/wrapit | /home/dogiermann/Repos/wrapit/lib/clang/13.0.1
Clang resource directory: /home/dogiermann/Repos/wrapit/lib/clang/13.0.1
WARNING: file '/home/dogiermann/Repos/wrapit/lib/clang/13.0.1/include/stddef.h' was not found. Please check the clang resource directory is set properly: see clang manual and use wrapit clang_opts parameter to pass options to clang.
Clang options:
    -x
    c++-header
    -std=c++17
    -I
    .
    -I
    /home/opt/llvm-dev/include/c++/v1/
    -resource-dir
    /home/dogiermann/Repos/wrapit/lib/clang/13.0.1

/usr/lib64/gcc/x86_64-pc-linux-gnu/13.2.1/../../../../include/c++/13.2.1/cstddef:50:10: 'stddef.h' file not found 
...

However, I am able to fix this problem locally if I compile wrapit with PIC enabled. Not really sure what exactly is going on. This seems to be reproducible by others https://stackoverflow.com/questions/24045131/why-does-dladdr-return-a-path-to-the-executable-on-position-independent-code-on .

grasph commented 7 months ago

Excellent finding! A bug of dladdr on Linux according to its man page.

I've modified CMakeLists.txt to compile with -fPIE, it's on the current HEAD 59cc196.

Let me know if it works out of the box now.

Thanks.

Philippe.

grasph commented 7 months ago

Discovery of the path at run time does not work with clang installed with Debian package system. Moved to a path discovered or set by the user at build time. Commit with this change: b2b9b0438b158a.

termi-official commented 7 months ago

The tests pass locally on my machine now (16184b8).

grasph commented 7 months ago

Thanks @termi-official for your feedback and all your help to fix the issue. I've changed the issue title and I will close it.

@bcicc if you have other problems with Ex002 example, please open a new issue.