JuliaInterop / Cxx.jl

The Julia C++ Interface
Other
757 stars 108 forks source link

LLVM error (TLS) #432

Open hros opened 5 years ago

hros commented 5 years ago

-- This issue appears in the discourse post -- I am wrapping a C++ math library (ntl). After successfully wrapping a major class, I encountered llvm errors in another class. The errors are: LLVM ERROR: Program used external function '__emutlsv.symbol' which could not be resolved! where symbol is replaced with the function symbol. checking the share library (libntl.so) the function is not missing. I am guessing this has to do with Thread Local Storage NTL has the following definition:

#ifdef NTL_THREADS

#define NTL_THREAD_LOCAL thread_local

#ifdef __GNUC__
#define NTL_CHEAP_THREAD_LOCAL __thread
#else
#define NTL_CHEAP_THREAD_LOCAL thread_local
#endif

#else

#define NTL_THREAD_LOCAL
#define NTL_CHEAP_THREAD_LOCAL

#endif

which means that certain elements in this class are prefixed with __thread and cannot be accessed using Cxx (as opposed to the other class I was able to wrap successfully)

Gnimuc commented 5 years ago

It looks like Emulated TLS is not enabled by default. Not sure whether this is the right way to fix the issue, but you could rebuild libcxxffi with Cxx->CI->getCodeGenOpts().EmulatedTLS= 1; and give it a try.

hros commented 5 years ago

Thanks for the suggestion. I was not able to fix the problem, however I may be doing something wrong. I wanted to make sure I was rebuilding libcxxffi, so I deleted it from the ...deps/use/lib/ directory (and verified it was gone) and then after issuing Cxx->CI->getCodeGenOpts().EmulatedTLS= 1; I rebuild Cxx (] build Cxx). The library was indeed rebuilt, however its timestamp was not current (it was the same timestamp of the file I have deleted)

Gnimuc commented 5 years ago

If you're not building Julia from source, Cxx.jl will always do a binary build which only downloads prebuilt libcxxffi binaries from here. That's why your modifications on bootstrap.cpp won't work.

Currently, those libcxxffi binaries are built using BB2 and here is the build script. It was a little bit complex. Even though the master branch now has a simplified version, but those dependent assets for Linux have not been uploaded yet.

Another way is to do a source build. You could build Julia from source with USE_BINARYBUILDER being disabled and then build Cxx. It's better to build Julia-v1.2, so two different Cxx versions won't mess up together.

hros commented 5 years ago

Thanks for your instructions. I was not able to create the library.

  1. I followed the instructions in your repo, modified the platforms array commenting all lines except the Linux:x86_64 line. Added the package BinaryBuilder, and then issued the command julia --color=yes build_tarballs.jl --verbose . The command was not successful. I'm attaching the log files. libcxxffi.log CMakeOutput.log
  2. I am not sure how to apply your suggestion of Cxx->CI->getCodeGenOpts().EmulatedTLS= 1; within this build process
  3. And lastly, once the libcxxffi.so is succesfully built, do I just overwrite the current version in my julia package directory?
Gnimuc commented 5 years ago

It looks like you didn't set up Julia and LLVM binary directories correctly. No worries, I just updated everything in this branch and now you can checkout that branch and run julia --color=yes build_tarballs.jl --verbose --debug x86_64-linux-gnu-gcc7.

I also did a local build on my virtual machine. It was built successfully but the audit complained about "cannot allocate memory in static TLS block" and there is no __emutls_v symbol in libcxxffi.

ERROR: could not load library "/home/parallels/Documents/CxxBuilder/build/x86_64-linux-gnu-gcc7/Aa2gUgP6/destdir/lib/clang/6.0.1/lib/linux/libclang_rt.scudo-x86_64.so"

/home/parallels/Documents/CxxBuilder/build/x86_64-linux-gnu-gcc7/Aa2gUgP6/destdir/lib/clang/6.0.1/lib/linux/libclang_rt.scudo-x86_64.so: cannot allocate memory in static TLS block