Open notinaboat opened 3 years ago
Better to rebuild the package upon the latest Clang.jl#master and use cross-platform support for loading those system headers. There might be compatibility issues between the stdlibs used in Julia and those on the users local machine.
Thanks for the tip @Gnimuc. In my current project I'm only using released packages, so I'll hold off using Clang.jl#master
until the next release.
@notinaboat I've been thinking of making a Julia package like rust's libc to provide similar functionalities in your package, which doesn't need libclang as a runtime dependency. But I don't have enough bandwidth to work on it.
Hi @Gnimuc, Out of interest, in CInclude.jl I've implemented a tiny inline C++ program to get macro-values: https://github.com/notinaboat/CInclude.jl/blob/master/src/CInclude.jl#L86-L92 This avoids having to parse and interpret C macro syntax.
Also of interest is this C REPL I found today: https://root.cern/cling/
Yeah. Clang is a full-featured compiler library with which you can do what a compiler can do, for example, implementing a C++ interpreter. https://github.com/JuliaInterop/Cxx.jl is another example.
Yes, Cxx.jl is awesome! I played with it a little many years ago, but I have never been able to use it in a project. It is very nice to see that you are working on it again :) I look forward to it being in-sync with latest Julia.
I've been thinking of making a Julia package like rust's libc to provide similar functionalities in your package...
A proof-of-concept of this is already demonstrated with https://github.com/analytech-solutions/System.jl using https://github.com/analytech-solutions/CBinding.jl to expose LibC and Linux API's, so it is possible to do in Julia.
Hi @Gnimuc, I'm in the process of updating my UnixIO.jl package to use the latest Clang.jl A few small changes to Clang.jl were needed to deal with wrapping system libraries. See: https://github.com/JuliaInterop/Clang.jl/pull/398
in CInclude.jl I've implemented a tiny inline C++ program to get macro-values. [...] Also of interest is this C REPL I found today: https://root.cern/cling/
The problem with trying to interpret the #define
values from Julia in generators/macro.jl
is that macro expressions can be arbitrarily complex and you'd end up having to build a complete C interpreter (libclang
only supports parsing C code to and AST, it does not provide an API for interpreting C expressions).
The problem with building an inline C++ program to evaluate#define
values is that if a single #define
resolves to something that does not have a value, the program will fail to compile.
I've now implemented a basic macro-value resolver using the C-REPL cling
.
(It would be nice to package cling
as a _jll
and call it from Clang.jl, but for now I'm just doing brew install cling
).
See: https://github.com/notinaboat/UnixIO.jl/blob/julia18/packages/UnixIOHeaders/src/UnixIOHeaders.jl#L65 This works by sending commands to the C-REPL on stdin, then parsing the results from stdout.
First the relevant headers are #include
-ed, then each #define
-ed symbol is entered into the REPL.
If the symbol resolves to something sensible the REPL prints out the C type and the value. e.g...
% cling -isystem/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk//usr/include
[cling]$ #include <signal.h>
[cling]$ SIGSEGV
(int) 11
[cling]$ SA_RESTART
(int) 2
[cling]$ SIG_DFL
(void (*)(int)) Function @0x0
[cling]$ BADSIG
(void (*)(int)) Function @0xffffffffffffffff
[cling]$ UINT8_MAX
(int) 255
[cling]$ UINT16_MAX
(int) 65535
[cling]$ UINT32_MAX
(unsigned int) 4294967295
[cling]$ UINT64_MAX
(unsigned long long) 18446744073709551615
In Julia we end up with:
const SIGSEGV = Cint(11)
const SA_RESTART = Cint(2)
const SIG_DFL = bitcast(Ptr{Cvoid}, UInt(0x00))
const BADSIG = bitcast(Ptr{Cvoid}, UInt(0xffffffffffffffff))
const UINT8_MAX = Cint(255)
const UINT16_MAX = Cint(65535)
const UINT32_MAX = Cuint(4294967295)
const UINT64_MAX = Culonglong(18446744073709551615)
The existing generators/macro.jl
produces this:
const SIGSEGV = 11
const SA_RESTART = 0x0002
const SIG_DFL = ((Cvoid(*))(Cint))(0)
const SIG_ERR = (Cvoid(*))(Cint) - 1
const BADSIG = SIG_ERR
const UINT8_MAX = 255
const UINT16_MAX = 65535
const UINT32_MAX = Cuint(4294967295)
const UINT64_MAX = Culonglong(18446744073709551615)
SIGSEGV
and SA_RESTART
have the wrong integer types (Int64
and UInt16
respectively)
SIG_DFL
and SIG_ERR
don't compile at all.
UINT8_MAX
and UINT16_MAX
are both Int64
where they should be Int32
In the case where the symbol does not resolve to a value, the cling REPL prints and error message. The output parser simply ignores symbols that result in an error message. e.g. ...
[cling]$ sa_handler
input_line_11:2:2: error: cannot use dot operator on a type
sa_handler
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk//usr/include/sys/signal.h:295:38: note: expanded from macro 'sa_handler'
#define sa_handler __sigaction_u.__sa_handler
^
[cling]$ PTHREAD_ONCE_INIT
input_line_19:2:2: error: expected ';' after expression
PTHREAD_ONCE_INIT
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk//usr/include/pthread.h:220:50: note: expanded from macro 'PTHREAD_ONCE_INIT'
#define PTHREAD_ONCE_INIT {_PTHREAD_ONCE_SIG_init, {0}}
^
input_line_19:2:2: error: expected ';' after expression
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk//usr/include/pthread.h:220:54: note: expanded from macro 'PTHREAD_ONCE_INIT'
#define PTHREAD_ONCE_INIT {_PTHREAD_ONCE_SIG_init, {0}}
^
[cling]$
and you'd end up having to build a complete C interpreter (
libclang
only supports parsing C code to and AST, it does not provide an API for interpreting C expressions).
As for the interpreter , some work has been done in https://github.com/Gnimuc/ClangCompiler.jl/pull/16, but not finished.
I'm still waiting for the integration of Cling and LLVM/Clang. There was a proposal for exposing a generic C++ interop interface from LLVM/Clang.
As for the interpreter , some work has been done in Gnimuc/ClangCompiler.jl#16, but not finished.
Looks good, I will look forward to this!
UINT8_MAX
andUINT16_MAX
are bothInt64
where they should beInt32
In Clang.jl, integer literals(e.g. 42
) use Int
as the default type. For other cases(e.g. 42LL
, 42ULL
), the type should be correctly matched. I don't remember why I did this though. 😅
Clang.jl also inherits Julia's behaviour of converting hexadecimal constants to the smallest possible unsigned type (In C they are int
). e.g. SA_RESTART
in the example above.
I'm wondering how bad this would be when doing interop-ing. I guess it's OK for most of the cases as long as the type is large enough to cover the value. But indeed this should be fixed in the future.
Cling is now available as a jll: https://github.com/JuliaBinaryWrappers/Cling_jll.jl Currently working on intel macOS and linux. Detail here: https://github.com/JuliaPackaging/Yggdrasil/blob/master/C/Cling/build_tarballs.jl#L102-L117
In some cases, especially for standard C library APIs, it is convenient to just
#include
a header without having to generate a wrapper first.I've made a little prototype
@cinclude
macro here: https://github.com/notinaboat/CInclude.jle.g.