Errors when compiling functions which depend on external libraries

[mdmaas]

Hi!

First of all, I want to mention that this is an amazing package!

Ok, so I was trying to compile a quadrature rule from FastGaussQuadrature, and got a segmentation fault.

So I ran a few tests... I eventually found that the dependency on SpecialFunctions was causing the problem.

using StaticCompiler
using SpecialFunctions
bessel_compiled, path = compile(besselj, (Float64,Complex{Float64}), "besselj" )

Now, the besselj function from bessel.jl in SpecialFunctions is essentially a wrapper to the AMOS library, which is accessed via ccall.

I continued testing if we can compile something which in turn calls an external library, and continued getting errors. For example:

using FFTW
compile(fft,(Vector{Complex{Float64}},))

leads, in the first place to a warning Found data we don't know how to relocate.. And then a compiler error:

MethodError: Cannot `convert` an object of type LLVM.ConstantExpr to an object of type Int64

I'll continue to add smaller and smaller tests, to see if this problem with external dependencies can be tackled somehow.

Best regards,

[mdmaas]

Ok, I have a minimal test that fails:

### Let's define a C function
using Libdl
C_code= """
       double mean(double a, double b) {
         return (a+b) / 2;
       }
       """

Clib=tempname()
open(`gcc -fPIC -O3 -xc -shared -o $(Clib * "." * Libdl.dlext) -`, "w") do f
    print(f, C_code)
end

cfunc(x,y)=ccall((:mean,Clib),Float64,(Float64,Float64), x, y)

### Compile cfunc
using StaticCompiler
compile(cfunc,(Float64,Float64))

I got the following error:

JIT session error: Symbols not found: [ mean ]
ERROR: LLVM error: Failed to materialize symbols: { (main, { julia_cfunc }) }

[brenhinkeller]

Hmm, interesting. I've noticed that ccalls don't play nice with standalone compilation (i.e., compile_executable, which comes with many more limitations than compile), but I thought that was just because ccall could introduce error handling / calls to libjulia.

You might try, just to see what happens, to load an external library using the dlopen, dlsym, and @ptrcall approach from StaticTools.jl (see e.g. this example) -- which should work even in static standalone binaries so ought (?) to work with compile too?

cc @MasonProtter as the expert on compile

[mdmaas]

Oh, I see. The docs in StaticTools state that ccall doesn't work smoothly with static compilation. So I'll give those methods a try.

If those methods work with StaticCompiler, we could then have a macro like @unsafe_ccall within compile, I guess, that could transform any occurrence of ccall to whatever we need.

[mdmaas]

Ok, I installed Julia 1.8 (the one I found on the home page, I'm not sure if that was 1.8.0-beta3), and tried:

libpath = "path_to_mylib.so"
lib = StaticTools.dlopen(libpath)
mean = StaticTools.dlsym(lib, c"mean")
a = 1.0
b = 2.2
pa, pb = pointer_from_objref(Ref(a)), pointer_from_objref(Ref(b))
@ptrcall mean(pa::Ptr{Nothing}, pb::Ptr{Nothing})::Float64

Got:

ERROR: `llvmcall` must be compiled to be called

[mdmaas]

Btw, I also can't run the example you linked to. I'm using julia-1.8.0-rc1.

[brenhinkeller]

That just means that code containing llvmcalls (which @ptrcall inserts) should be wrapped in a function. Base.llvmcall does indeed need to be compiled to be called, and putting it in a function is usually the easiest way to do that.

[mdmaas]

Oh, that's the first time I've seen this error message...

I've got to compile call_mean as a shared library now, thanks!

I couldn't run the example in the docs, though, dlmul() doesn't run for me (let alone compiling it).

I also noticed that instead of StaticTools.dlsym(lib, c"mean") I need Libdl.dlsym(lib, "mean").

I guess that if we could manage to bypass ccall with these kinds of wrappers, then we could compile most of the math functions which depend on FFTW, or SpecialFunction, and so on.

[mdmaas]

Now, what would be the best way to make this work for us without having to create a dev version of any package that uses ccall, and change it manually?

I'm quite new to macros, but would it be possible to write one that would replace any occurrence of ccall with a call to a certain static_ccall?

[brenhinkeller]

Hm, that's odd about StaticTools.dlsym vs Libdl.dlsym -- mind opening an issue on StaticTools for that?

In principle one could do that sort of thing with perhaps macros or perhaps at the IR level with something like Casette or Mixtape like is being considered in https://github.com/tshort/StaticCompiler.jl/pull/69 -- but I think there's also some hope that we'll with a bit more work be able to support things like this natively in compile if not in compile_executable

[mdmaas]

Here's the Issue in StaticTools

From what I read in #69, it looks like Mixtape doesn't work yet with Julia 1.8, and that implementing this functionality at the GPUCompiler level won't add much to what Mixtape is already doing. So I guess fixing Mixtape is the way to go. I'd need to learn how to use it first, as well.

Btw, I'm very interested in both static compilation and GPUs, and it was nice finding out that these things run on the same infrastructure :+1:.