Cross-compile to Windows (64-bit) is failing given that OpenSSL can't be found

[TSC21]

I am trying to cross-compile libdatachannel to Windows using dockcross (more specifically, https://github.com/dockcross/dockcross/tree/master/windows-static-x64) in Ubuntu 20.04. This container image in specific uses MXE together with the x86_64-w64-mingw32 compiler.

OpenSSL is also built through this container using the following configuration and build command: (note this is being called from CMake, and so the usage of CMake variables)

# Config
perl <SOURCE_DIR>/Configure mingw64 no-shared no-tests --prefix=${CMAKE_INSTALL_PREFIX}
# build
RC=/usr/src/mxe/usr/bin/x86_64-w64-mingw32.static-windres
AR=/usr/src/mxe/usr/bin/x86_64-w64-mingw32.static-ar
RANLIB=/usr/src/mxe/usr/bin/x86_64-w64-mingw32.static-ranlib
CROSS_COMPILE=x86_64-w64-mingw32.static-
make -e RC=${RC} AR=${AR} RANLIB=${RANLIB} CROSS_COMPILE=${CROSS_COMPILE}

The command above runs properly and libcrypto.a and libssl.a are both installed in ${CMAKE_INSTALL_PREFIX}/lib, besides the headers on the ${CMAKE_INSTALL_PREFIX}/include folder. Here's the install tree structure:

.
├── bin
│   ├── ...
│   └── openssl.exe
├── include
│   ├── ...
│   └── openssl
│      ├── ...
│      └── ssl.h
├── lib
│   ├── ...
│   ├── libcrypto.a
│   └── libssl.a
...

But, for some reason, libdatachannel when calling find_package(OpenSSL REQUIRED), fails to find OPENSSL_CRYPTO_LIBRARY as it can be seen above. Note that OPENSSL_INCLUDE_DIR had to also be passed as a CMAKE_ARG to build lidatachannel because FindOpenSSL() is was also not able to find openssl/ssl.h, although it is installed in ${CMAKE_INSTALL_PREFIX}/include. OpenSSL is marked as a dependency of libdatachannel while building through CMake, so I don't necessarily understand why this fails. One thing to note is that MXE sets MSYS to 1, and also since we are using mingw32 to cross-compile,WIN32 and MINGW are also set to 1. If I try to manually set MINGW and WIN32 to 0, then find_package(OpenSSL) succeeds and the build process continues, until the linker fails because the architecture is of course wrong.

This is the error seen:

-- Compiler flags (CMAKE_C_FLAGS):  -std=c99 -pedantic -Wall -Wextra -Wfloat-equal -Wshadow -Wpointer-arith -Wunreachable-code -Winit-self -Wno-unused-function -Wno-unused-parameter -Wno-unreachable-code -Wstrict-prototypes -Werror
-- link library: ws2_32
-- Configuring done
CMake Error at /usr/src/mxe/usr/x86_64-pc-linux-gnu/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the
  system variable OPENSSL_ROOT_DIR (missing: OPENSSL_CRYPTO_LIBRARY) (found
  version "1.1.1o")
Call Stack (most recent call first):
  /usr/src/mxe/usr/x86_64-pc-linux-gnu/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  /usr/src/mxe/usr/x86_64-pc-linux-gnu/share/cmake-3.20/Modules/FindOpenSSL.cmake:571 (find_package_handle_standard_args)
  deps/libsrtp/CMakeLists.txt:75 (find_package)

Is this potentially a problem with the OpenSSL build or even with the default FindOpenSSL.cmake module? Or is this something that can be addressed on libdatachannel CMakeLists.txt? Any help with this would be appreciated.

Thanks in advance!

[paullouisageneau]

But, for some reason, libdatachannel when calling find_package(OpenSSL REQUIRED), fails to find OPENSSL_CRYPTO_LIBRARY as it can be seen above. Note that OPENSSL_INCLUDE_DIR had to also be passed as a CMAKE_ARG to build lidatachannel because FindOpenSSL() is was also not able to find openssl/ssl.h, although it is installed in ${CMAKE_INSTALL_PREFIX}/include.

OPENSSL_INCLUDE_DIR is actually an output variable of FindOpenSSL(), so you must not set it yourself. Instead, you should set OPENSSL_ROOT_DIR, and OPENSSL_USE_STATIC_LIBS to TRUE since the OpenSSL build is static.

[TSC21]

But, for some reason, libdatachannel when calling find_package(OpenSSL REQUIRED), fails to find OPENSSL_CRYPTO_LIBRARY as it can be seen above. Note that OPENSSL_INCLUDE_DIR had to also be passed as a CMAKE_ARG to build lidatachannel because FindOpenSSL() is was also not able to find openssl/ssl.h, although it is installed in ${CMAKE_INSTALL_PREFIX}/include.

OPENSSL_INCLUDE_DIR is actually an output variable of FindOpenSSL(), so you must not set it yourself. Instead, you should set OPENSSL_ROOT_DIR, and OPENSSL_USE_STATIC_LIBS to TRUE since the OpenSSL build is static.

@paullouisageneau yes I know the above and the only reason I set OPENSSL_INCLUDE_DIR is because I saw it at least allowed to find the include dirs - it was just something I tried. It doesn't matter if I set OPENSSL_USE_STATIC_LIBS - it's still not able to find it. It seems to be though this might be a problem with libsrtp, since the error is coming from there (still, it uses find_package(OpenSSL REQUIRED) as in libdatachannel).

[jazzcake]

One more tip:

After installing openssl (in my case I set environment variables: OPENSSL_CONF, OPENSSL_HOME in this link).

Then open the last version of Visual Studio Community and use "Open local folder" to specify the libdatachannel project.

Visual Studio Community automatically builds the project without any problems. I was getting multiple link errors with nmake & Makefiles but Visual Studio fixed them all at once.

[Tim-S]

I guess the issue is already solved, if not, try setting OPENSSL_ROOT_DIR and force -D CMAKE_POLICY_DEFAULT_CMP0074=NEW (had a smilar issue when compiling libdatachannel for windows with mbedtls instead of openssl in https://gitlab.com/Tim-S/libdatachannel-win32-builds)