Error when compiling to musl target

[bitcapybara]

Hi, The latest 0.19.0 version will fail when compiling to musl target, I don't know if it is caused by my environment.

step

git submodule init
git submodule update
cargo clean
YARA_CRYPTO_LIB=disable cargo run --target=x86_64-unknown-linux-musl --example tutorial --features yara-sys/vendored

output

The following warnings were emitted during compilation:

warning: /path/to/yara-rust/target/x86_64-unknown-linux-musl/debug/build/yara-sys-ec797a34daf20117/out/yara/libyara/filemap.c:162:10: fatal error: linux/magic.h: No such file or directory
warning:   162 | #include <linux/magic.h>
warning:       |          ^~~~~~~~~~~~~~~
warning: compilation terminated.

error: failed to run custom build command for `yara-sys v0.19.0 (/path/to/yara-rust/yara-sys)`

tried

It works fine if I use 0.17.0 and I don't know how to fix this error, I installed linux-headers and file package but it didn't work

environment

Archlinux WSL2

uname -a
Linux yanyuxing 5.15.90.1-microsoft-standard-WSL2 #1 SMP Fri Jan 27 02:56:13 UTC 2023 x86_64 GNU/Linux

yara --version
4.3.0

ls -al /lib/libyara.so
lrwxrwxrwx 1 root root 17 Apr 20 05:07 /lib/libyara.so -> libyara.so.10.0.0*

[Hugal31]

This line was added to Yara in v4.3.0. I don't know musl, but you could find a way to add linux/magic.h to your system. On my Ubuntu, it is installed with the libc (which you don't want) and with the linux-headers-* packages.

This is rather a problem with Yara itself than with yara-rust.

[bitcapybara]

When the target is not musl, everything works fine.

I also tried compiling the project in Alpine image, this error also occurs, but I can find the file at /usr/include/magic.h

[ikrivosheev]

When the target is not musl, everything works fine.

I also tried compiled the project in Alpine image, this error also occurs, but I can find the file at /usr/include/magic.h

This had been added: https://github.com/VirusTotal/yara/pull/1848

[bitcapybara]

In my archlinux computer, I can find magic.h:

ll /usr/include/linux/magic.h
-rw-r--r-- 1 root root 3860 Feb  1 20:56 /usr/include/linux/magic.h

But this error still occurs

[Hugal31]

Can you build Yara using musl?

[bitcapybara]

I tried this in Alpine docker image, which has a musl environment:

steps

1. docker run -it --rm alpine:3.17 /bin/sh

(inside alpine)
1. apk update && apk add build-base automake libtool make gcc pkgconfig autoconf linux-headers
2. cd
3. wget https://github.com/VirusTotal/yara/archive/refs/tags/v4.3.0.tar.gz
4. tar -xvf v4.3.0.tar.gz
5. cd yara-4.3.0/
6. ./bootstrap.sh
7. ./configure
8. make
9. make install
10. make check

result

$ ldd $(which yara)
        /lib/ld-musl-x86_64.so.1 (0x7f0aaf817000)
        libyara.so.10 => /usr/local/lib/libyara.so.10 (0x7f0aaf78d000)
        libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7f0aaf817000)

Then I build a new docker image with Dockerfile:

FROM docker.io/library/rust:1.69-alpine

WORKDIR $HOME

RUN apk update && apk add build-base perl cmake automake libtool make gcc pkgconfig autoconf linux-headers && \
    wget https://github.com/VirusTotal/yara/archive/refs/tags/v4.3.0.tar.gz && \
    tar -xvf v4.3.0.tar.gz && cd yara-4.3.0/ && \
    ./bootstrap.sh && ./configure && make && make install && make check && \
    rustup component add rustfmt clippy

compile yara-rust with new image:

$ docker run -it --rm -v $(pwd):/app -w /app rust/yara:4.3.0 /bin/sh

$ (inside alpine) YARA_CRYPTO_LIB=disable cargo run --example tutorial --features yara-sys/vendored

another error occurs:

  running: "ar" "s" "/app/target/debug/build/yara-sys-f319065f8e94f08b/out/libyara.a"
  exit status: 0
  cargo:rustc-link-lib=static=yara
  cargo:rustc-link-search=native=/app/target/debug/build/yara-sys-f319065f8e94f08b/out
  cargo:rerun-if-env-changed=YARA_DEBUG_VERBOSITY
  cargo:rerun-if-env-changed=YARA_DEBUG_VERBOSITY_x86_64-unknown-linux-musl
  cargo:rerun-if-env-changed=YARA_DEBUG_VERBOSITY_x86_64_unknown_linux_musl
  cargo:rerun-if-env-changed=YARA_OPENSSL_DIR
  cargo:rerun-if-env-changed=YARA_OPENSSL_DIR_x86_64-unknown-linux-musl
  cargo:rerun-if-env-changed=YARA_OPENSSL_DIR_x86_64_unknown_linux_musl
  cargo:rerun-if-env-changed=YARA_OPENSSL_LIB_DIR
  cargo:rerun-if-env-changed=YARA_OPENSSL_LIB_DIR_x86_64-unknown-linux-musl
  cargo:rerun-if-env-changed=YARA_OPENSSL_LIB_DIR_x86_64_unknown_linux_musl
  cargo:rerun-if-env-changed=YARA_OPENSSL_INCLUDE_DIR
  cargo:rerun-if-env-changed=YARA_OPENSSL_INCLUDE_DIR_x86_64-unknown-linux-musl
  cargo:rerun-if-env-changed=YARA_OPENSSL_INCLUDE_DIR_x86_64_unknown_linux_musl
  cargo:rerun-if-env-changed=YARA_LIBRARY_PATH
  cargo:rerun-if-env-changed=YARA_LIBRARY_PATH_x86_64-unknown-linux-musl
  cargo:rerun-if-env-changed=YARA_LIBRARY_PATH_x86_64_unknown_linux_musl
  cargo:rerun-if-env-changed=YARA_CRYPTO_LIB
  cargo:rerun-if-env-changed=YARA_CRYPTO_LIB_x86_64-unknown-linux-musl
  cargo:rerun-if-env-changed=YARA_CRYPTO_LIB_x86_64_unknown_linux_musl
  cargo:rustc-link-search=native=/app/target/debug/build/yara-sys-f319065f8e94f08b/out
  cargo:rustc-link-lib=static=yara
  cargo:include=/app/target/debug/build/yara-sys-f319065f8e94f08b/out/yara/libyara/include
  cargo:lib=/app/target/debug/build/yara-sys-f319065f8e94f08b/out

  --- stderr
  thread 'main' panicked at 'Unable to find libclang: "couldn't find any valid shared libraries matching: ['libclang.so', 'libclang-*.so', 'libclang.so.*', 'libclang-*.so.*'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [])"', /usr/local/cargo/registry/src/rsproxy.cn-0dccff568467c15b/bindgen-0.64.0/./lib.rs:2393:31
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

So I installed clang-dev, met this error:

$ apk add clang-dev
$ YARA_CRYPTO_LIB=disable cargo run --example tutorial --features yara-sys/vendored

thread 'main' panicked at 'Unable to find libclang: "the `libclang` shared library at /usr/lib/libclang.so.15.0.7 could not be opened: Dynamic loading not supported"', /usr/local/cargo/registry/src/rsproxy.cn-0dccff568467c15b/bindgen-0.64.0/./lib.rs:2393:31
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

What confuses me is that it doesn't try to find the static clang library

[bitcapybara]

hello! I fixed this problem on my ArchLinux by installing kernel-headers-musl package, and on debian bullseye docker image fixed by ln -s /usr/include/linux /usr/include/x86_64-linux-musl/linux, I think we can close this issue now, 😄

this is my debian docker file:

FROM docker.io/library/rust:1.71-bullseye

RUN apt-get update && apt-get install -y musl musl-tools clang && rm -rf /var/lib/apt/lists/* && \
    rustup target add x86_64-unknown-linux-musl && rustup component add rustfmt clippy && \
    ln -s /usr/include/linux /usr/include/x86_64-linux-musl/linux

@Hugal31 @ikrivosheev

[ikrivosheev]

@bitcapybara hello! I think yes, can you close the issue?