JuliaInterop / Clang.jl

C binding generator and Julia interface to libclang
https://juliainterop.github.io/Clang.jl/
MIT License
219 stars 68 forks source link

Unix C library compatibility #398

Closed notinaboat closed 1 year ago

notinaboat commented 1 year ago

Changes to enable wrapping of the system C library on mac OS and Linux.

See commit logs for detail.

Currently I am wrapping: <errno.h>, <string.h>, <limits.h>, <stdlib.h>, <pthread.h>, <sys/ioctl.h>, <termios.h>, <fcntl.h>, <poll.h>, <sys/epoll.h> <unistd.h>, <sys/stat.h>, <net/if.h>, <sys/socket.h>, <signal.h>, <sys/wait.h>, <sys/syscall.h> and <spawn.h>.

... and i'm using these options:

   Dict{String,Any}(
            "general" => Dict{String,Any}(
                "is_local_header_only" => false,
                "auto_mutability" => true,
                "auto_mutability_includelist" => ["termios"],
                "library_name" => ""
            ),
            "codegen" => Dict{String,Any}(
                "use_ccall_macro" => true,
                "wrap_variadic_function" => false
            ),
            "codegen.macro" => Dict{String,Any}(
                "macro_mode" => "aggressive",
                "ignore_pure_definition" => true
            )
        )

See https://github.com/notinaboat/UnixIO.jl/blob/c03f1695999666ab95177b5e1713e2fe1cd1eae0/packages/UnixIOHeaders/src/UnixIOHeaders.jl#L162-L188

Gnimuc commented 1 year ago

Is there any reason why you choose not to use Ygg system headers?

notinaboat commented 1 year ago

Is there any reason why you choose not to use Ygg system headers?

Those headers appear to be out of date.

For example, I use the POSIX spawn interface here: https://github.com/notinaboat/UnixIO.jl/blob/julia18/src/UnixIO.jl#L1282-L1284

But, the POSIX_SPAWN_SETSID symbol is not defined in the Ygg headers...

julia> using Clang

julia> dirs =Clang.JLLEnvs.get_system_dirs("x86_64-apple-darwin14")
5-element Vector{String}:
 "/Users/samoconnor/.julia/artifa" ⋯ 57 bytes ⋯ "64-apple-darwin14/4.8.5/include"
 "/Users/samoconnor/.julia/artifa" ⋯ 63 bytes ⋯ "le-darwin14/4.8.5/include-fixed"
 "/Users/samoconnor/.julia/artifa" ⋯ 43 bytes ⋯ "9/x86_64-apple-darwin14/include"
 "/Users/samoconnor/.julia/artifa" ⋯ 56 bytes ⋯ "e-darwin14/sys-root/usr/include"
 "/Users/samoconnor/.julia/artifa" ⋯ 70 bytes ⋯ "-root/System/Library/Frameworks"

julia> dirs[1]
"/Users/samoconnor/.julia/artifacts/303e65f6896f295459c81aa311b5eb993f702d89/lib/gcc/x86_64-apple-darwin14/4.8.5/include"

e.g. the file spawn.h is found...

% find /Users/samoconnor/.julia/artifacts/303e65f6896f295459c81aa311b5eb993f702d89 -name spawn.h                                                    
/Users/samoconnor/.julia/artifacts/303e65f6896f295459c81aa311b5eb993f702d89/x86_64-apple-darwin14/sys-root/usr/include/spawn.h
...

But the symbol POSIX_SPAWN_SETSID is not found...

% find /Users/samoconnor/.julia/artifacts/303e65f6896f295459c81aa311b5eb993f702d89 -name spawn.h | xargs grep POSIX_SPAWN_SETSID
% 

Compare to searching under Applications/Xcode.app ...

% find /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/ -name spawn.h | xargs grep POSIX_SPAWN_SETSID
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk//usr/include/sys/spawn.h:#define POSIX_SPAWN_SETSID              0x0400
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk//System/Library/Frameworks/Kernel.framework/Versions/A/Headers/sys/spawn.h:#define POSIX_SPAWN_SETSID              0x0400
notinaboat commented 1 year ago

... also, I'm a little uneasy about using platform headers from https://github.com/phracker/MacOSX-SDKs I would rather use official headers form the platform vendor.

notinaboat commented 1 year ago

Background FYI:

My motivation for creating UnixIO.jl came from working on an embedded system (R'Pi-linux) and having trouble with unreliable serial port IO. I could write a tiny C program to open /dev/ttyS0 and read/write with 100% reliability, but from Julia it was unreliable. I started trying to find the root cause, but that was complicated by all the intermediate layers in Base.IO and libuv. It seems that the libuv project simply does really intend to support serial ports: https://github.com/libuv/libuv/issues/1970

So, the intention of UnixIO.jl is:

Gnimuc commented 1 year ago

Those headers appear to be out of date.

For example, I use the POSIX spawn interface here: https://github.com/notinaboat/UnixIO.jl/blob/julia18/src/UnixIO.jl#L1282-L1284

But, the POSIX_SPAWN_SETSID symbol is not defined in the Ygg headers...

The default gcc version used in Clang.jl is v0.4.8.5(along with those system headers), which can be updated to a newer one. But in your use case, I think it's better to just use headers on your local system.

Background FYI:

My motivation for creating UnixIO.jl came from working on an embedded system (R'Pi-linux) and having trouble with unreliable serial port IO. I could write a tiny C program to open /dev/ttyS0 and read/write with 100% reliability, but from Julia it was unreliable. I started trying to find the root cause, but that was complicated by all the intermediate layers in Base.IO and libuv. It seems that the libuv project simply does really intend to support serial ports: libuv/libuv#1970

So, the intention of UnixIO.jl is:

  • I just want to talk directly to the native OS C library on my machine.
  • I want to avoid any risk that might be introduced by 3rd party intermediate layers.

Makes sense to me now. Thanks for the PR. :)