libyara supports three mutually exclusive crypto backends that it can use in its
hash and pe modules, which it chooses during compilation: OpenSSL, WinCrypt (windows)
or Common Crypto (macos). The original configure.ac is in charge of finding the right crypto backend.
I'm targeting windows platforms where OpenSSL is not installed but wincrypt is, so here's a PR that adds support for it.
The current build.rs only supported dynamic linking against OpenSSL, and dynamic linking against OpenSSL on windows platforms (e.g. in a MinGW environment). I'm adding support for dynamic linking against WinCrypt or Common Crypto instead, which are always installed on windows and macos.
Beside, the current build.rs had a few problems in cross-compilation scenarios:
tries to dl_open libcrypto (OpenSSL) during build.rs and exits if not found, therefore does not support simple cross compilation scenarios where openssl is installed on the target but not the host.
OPENSSL_LIB_DIR does not influence the dl_open search path, so it checks for the presence of the wrong lib.
Wincrypt / CommonCrypto are not supported: it tries to dl_open OpenSSL inconditonally, and exits.
Supports openssl on windows (e.g. MinGW like environments) or macos, but sets both -D HAVE_LIBCRYPTO and -D HAVE_WINCRYPT_H or -D HAVE_COMMONCRYPTO_COMMONCRYPTO_H. These are all mutually exclusive flags.
For these reasons, I've removed the Library::new() check which verifies if the lib can be found at build time, because in my opinion it's no longer applicable when the host and target platform are different, and we should let cargo fail at link time when the lib we specified via a cargo:rustc-link-lib=dylib=libXXX was nowhere to be found.
I've changed the YARA_ENABLE_CRYPTO boolean flag to YARA_CRYPTO_LIB which specifies the
desired crypto backend. When unspecified, we default to the crypto backend of
the target os, which will always be available. The user can still override
this default, for exemple to force the use of OpenSSL on a windows target that
has a MinGW-like environment.
Because it's tedious to have to set and unset this env var (and others) when switching between two different targets, I've adopted the model that CC uses for its env vars: it looks first for <var_name>_<target> then for <var_name>_<target_with_underscores> and only then for <var_name>. This way you can define things like this in your environment:
YARA_CRYPTO_LIB_x86_64_unknown_linux_gnu=OpenSSL
YARA_CRYPTO_LIB_x86_64_pc_windows_gnu=OpenSSL # targeting MinGW env
YARA_CRYPTO_LIB_x86_64_pc_windows_msvc=WinCrypt # targeting windows
Because we now have a crypto lib by default, I've changed the hash module compilation to default to Enable, like in the original libyara.
For macos, I had to change the github action workflow's openssl dir to /usr/local/opt/openssl@1.1/ because it failed in my fork, and when debugging with action-tmate, the /usr/local/opt/openssl symlink was missing in my VM. I don't know how it worked for your previous runs 🤷♀
Hugal31
commented
2 years ago
libyara supports three mutually exclusive crypto backends that it can use in its hash and pe modules, which it chooses during compilation: OpenSSL, WinCrypt (windows) or Common Crypto (macos). The original configure.ac is in charge of finding the right crypto backend.
I'm targeting windows platforms where OpenSSL is not installed but wincrypt is, so here's a PR that adds support for it.
The current
build.rsonly supported dynamic linking against OpenSSL, and dynamic linking against OpenSSL on windows platforms (e.g. in a MinGW environment). I'm adding support for dynamic linking against WinCrypt or Common Crypto instead, which are always installed on windows and macos.Beside, the current build.rs had a few problems in cross-compilation scenarios:
OPENSSL_LIB_DIRdoes not influence the dl_open search path, so it checks for the presence of the wrong lib.-D HAVE_LIBCRYPTOand-D HAVE_WINCRYPT_Hor-D HAVE_COMMONCRYPTO_COMMONCRYPTO_H. These are all mutually exclusive flags.For these reasons, I've removed the
Library::new()check which verifies if the lib can be found at build time, because in my opinion it's no longer applicable when the host and target platform are different, and we should let cargo fail at link time when the lib we specified via acargo:rustc-link-lib=dylib=libXXXwas nowhere to be found.I've changed the YARA_ENABLE_CRYPTO boolean flag to YARA_CRYPTO_LIB which specifies the desired crypto backend. When unspecified, we default to the crypto backend of the target os, which will always be available. The user can still override this default, for exemple to force the use of OpenSSL on a windows target that has a MinGW-like environment.
Because it's tedious to have to set and unset this env var (and others) when switching between two different targets, I've adopted the model that CC uses for its env vars: it looks first for
<var_name>_<target>then for<var_name>_<target_with_underscores>and only then for<var_name>. This way you can define things like this in your environment:Because we now have a crypto lib by default, I've changed the hash module compilation to default to Enable, like in the original libyara.
For macos, I had to change the github action workflow's openssl dir to
/usr/local/opt/openssl@1.1/because it failed in my fork, and when debugging with action-tmate, the/usr/local/opt/opensslsymlink was missing in my VM. I don't know how it worked for your previous runs 🤷♀