zzblydia
commented
7 months ago Could you show the output log when you execute cmake? Do you have two different versions of the OpenSSL library in your environment?
Open memsharded opened 1 year ago
zzblydia
commented
7 months ago Could you show the output log when you execute cmake? Do you have two different versions of the OpenSSL library in your environment?
memsharded
commented
7 months ago Yes, I have different openssl versions built locally, in Conan packages. Everything is exactly the same, but with different openssl version. I am launching the build from https://github.com/conan-io/conan-center-index/tree/master/recipes/libwebsockets/all, with:
conan create . --version=4.3.2 2> build.logYes, I have different openssl versions built locally, in Conan packages. Everything is exactly the same, but with different openssl version. I am launching the build from https://github.com/conan-io/conan-center-index/tree/master/recipes/libwebsockets/all, with:
conan create . --version=4.3.2 2> build.logBuilding against openssl/1.1.1w (success) Building against openssl/3.1.2 (changing the conanfile.py
requires) (error)
I tested compiling libwebsockets4.3.2 with OpenSSL 3.0.12 (without conan) successfully on Windows 10 and Ubuntu 22.
lws defines the macro LWS_HAVE_HMAC_CTX_new in tls/CMakeLists.txt after finding HMAC_CTX_new. I found "Looking for HMAC_CTX_new - not found" in your 3.1.2 build log, while "Looking for HMAC_CTX_new - found" in your 1.1.1w build log. This leads to the error you encountered: 'error: field 'ctx' has incomplete type'.
The HMAC_CTX_new function has been deprecated since OpenSSL 3.0. However, it maintains compatibility by allowing HMAC_CTX_new to be used with setting OPENSSL_API_COMPAT to 30000 by default. Therefore, I believe the issue may be caused by the downloaded libcrypto.so of version 3.1.2.
To further investigate, I reproduced the issue on Windows 10 using Conan 2 and VS2022. I performed the build using OpenSSL versions 1.1.1w and 3.0.12. When I placed the libcrypto.lib file from 1.1.1w into the OpenSSL lib folder of version 3.0.12 and recompiled, it showed "Looking for HMAC_CTX_new - found,". So I believe it is worth trying to check the downloaded OpenSSL 3.1.2 library to resolve the issue.
lws-team
commented
7 months ago @zzblydia thanks for sharing your findings.
zzblydia
commented
6 months ago @lws-team it's my pleasure. @memsharded I believe I have identified the root cause of the issue.
I wrote a demo.c file that calls the function HMAC_CTX_new and attempted to link the OpenSSL library (3.1.2) that your project download. However, Visual Studio 2022 displayed an error: 'LNK2019 unresolved external symbol deflate referenced in function zlib_stateful_compress_block ... libcrypto.lib(libcrypto-lib-c_zlib.obj)'.
After investigating, I discovered that the OpenSSL library (libcrypto) from your project download has a dependency on zlib. This caused the CHECK_FUNCTION_EXISTS(${VARIA}HMAC_CTX_new LWS_HAVE_HMAC_CTX_new PARENT_SCOPE) in tls/CMakeLists.txt to return 'not found'.
I suggest setting 'with_zlib': 'zlib' (default_options) in conanfile.py, as the OpenSSL library(3.1.2 or 3.0.12) downloaded relies on zlib. I tested this solution with OpenSSL 3.1.2 (and 3.0.12) and found it to be effective.
cmake -G "Visual Studio 17 2022" -DCMAKE_TOOLCHAIN_FILE="C:/Users/vmwin10/.conan2/p/b/libwefb6db7f5b2349/b/build/generators/conan_toolchain.cmake" -DCMAKE_INSTALL_PREFIX="C:/Users/vmwin10/.conan2/p/b/libwefb6db7f5b2349/p" -DCMAKE_POLICY_DEFAULT_CMP0091="NEW" "C:\Users\vmwin10.conan2\p\b\libwefb6db7f5b2349\b\src"
-- Performing Test LWS_HAVE_SUSECONDS_T -- Performing Test LWS_HAVE_SUSECONDS_T - Failed zlib/miniz include dirs: C:/Users/vmwin10/.conan2/p/zlibee1f000851145/p/include zlib/miniz libraries: C:/Users/vmwin10/.conan2/p/zlibee1f000851145/p/lib/zlib.lib -- Performing Test LWS_HAVE_PIPE2 -- Performing Test LWS_HAVE_PIPE2 - Failed -- Performing Test LWS_HAVE_TCP_USER_TIMEOUT -- Performing Test LWS_HAVE_TCP_USER_TIMEOUT - Failed Compiling with SSL support OpenSSL include dir: C:/Users/vmwin10/.conan2/p/opens7eaedae154df5/p/include OpenSSL libraries: C:/Users/vmwin10/.conan2/p/opens7eaedae154df5/p/lib/libssl.lib;C:/Users/vmwin10/.conan2/p/opens7eaedae154df5/p/lib/libcrypto.lib -- Looking for openssl/ecdh.h -- Looking for openssl/ecdh.h - not found CHECK_FUNCTION_EXISTS CMAKE_REQUIRED_INCLUDES: C:/Users/vmwin10/.conan2/p/opens7eaedae154df5/p/include CHECK_FUNCTION_EXISTS CMAKE_REQUIRED_LIBRARIES: C:/Users/vmwin10/.conan2/p/opens7eaedae154df5/p/lib/libssl.lib;C:/Users/vmwin10/.conan2/p/opens7eaedae154df5/p/lib/libcrypto.lib;C:/Users/vmwin10/.conan2/p/zlibee1f000851145/p/lib/zlib.lib;ws2_32.lib;userenv.lib;psapi.lib;iphlpapi.lib;crypt32.lib;C:/Users/vmwin10/.conan2/p/zlibee1f000851145/p/lib/zlib.lib -- Looking for SSL_CTX_set1_param -- Looking for SSL_CTX_set1_param - found -- Looking for SSL_set_info_callback -- Looking for SSL_set_info_callback - found -- Looking for X509_VERIFY_PARAM_set1_host -- Looking for X509_VERIFY_PARAM_set1_host - found -- Looking for X509_VERIFY_PARAM_set1_host -- Looking for X509_VERIFY_PARAM_set1_host - not found -- Looking for RSA_set0_key -- Looking for RSA_set0_key - found -- Looking for X509_get_key_usage -- Looking for X509_get_key_usage - found -- Looking for SSL_CTX_EVP_PKEY_new_raw_private_key -- Looking for SSL_CTX_EVP_PKEY_new_raw_private_key - not found -- Looking for SSL_CTX_get0_certificate -- Looking for SSL_CTX_get0_certificate - found -- Looking for SSL_get0_alpn_selected -- Looking for SSL_get0_alpn_selected - found -- Looking for SSL_set_alpn_protos -- Looking for SSL_set_alpn_protos - found -- Looking for EVP_aes_128_cfb8 -- Looking for EVP_aes_128_cfb8 - found -- Looking for EVP_aes_128_cfb128 -- Looking for EVP_aes_128_cfb128 - found -- Looking for EVP_aes_192_cfb8 -- Looking for EVP_aes_192_cfb8 - found -- Looking for EVP_aes_192_cfb128 -- Looking for EVP_aes_192_cfb128 - found -- Looking for EVP_aes_256_cfb8 -- Looking for EVP_aes_256_cfb8 - found -- Looking for EVP_aes_256_cfb128 -- Looking for EVP_aes_256_cfb128 - found -- Looking for EVP_aes_128_xts -- Looking for EVP_aes_128_xts - found -- Looking for EVP_aes_128_ofb -- Looking for EVP_aes_128_ofb - found -- Looking for EVP_aes_128_ecb -- Looking for EVP_aes_128_ecb - found -- Looking for EVP_aes_128_ctr -- Looking for EVP_aes_128_ctr - found -- Looking for RSA_verify_pss_mgf1 -- Looking for RSA_verify_pss_mgf1 - not found -- Looking for HMAC_CTX_new -- Looking for HMAC_CTX_new - found -- Looking for EVP_PKEY_new_raw_private_key -- Looking for EVP_PKEY_new_raw_private_key - found -- Looking for SSL_SESSION_set_time -- Looking for SSL_SESSION_set_time - found -- Looking for EC_KEY_new_by_curve_name -- Looking for EC_KEY_new_by_curve_name - found -- Performing Test LWS_HAVE_SSL_EXTRA_CHAIN_CERTS -- Performing Test LWS_HAVE_SSL_EXTRA_CHAIN_CERTS - Success -- Performing Test LWS_HAVE_EVP_MD_CTX_free -- Performing Test LWS_HAVE_EVP_MD_CTX_free - Success -- Performing Test LWS_HAVE_OPENSSL_STACK -- Performing Test LWS_HAVE_OPENSSL_STACK - Success -- Looking for ECDSA_SIG_set0 -- Looking for ECDSA_SIG_set0 - found -- Looking for BN_bn2binpad -- Looking for BN_bn2binpad - found -- Looking for EVP_aes_128_wrap -- Looking for EVP_aes_128_wrap - found -- Looking for EC_POINT_get_affine_coordinates -- Looking for EC_POINT_get_affine_coordinates - found -- Looking for TLS_client_method -- Looking for TLS_client_method - found -- Looking for TLSv1_2_client_method -- Looking for TLSv1_2_client_method - found `
ydcpp
commented
2 months ago Hi, I was also having this issue with conan build. When I build openssl 3 as shared library in conan dependencies there is no problem with building LWS.
But when openssl is built as static library for version 3xx , then LWS CMake cannot locate some of the new functions of openssl. Then fallbacks to old openssl definitions into compilation errors.
I'm not sure if this issue is related to LWS configurations or openssl itself with static build.
memsharded
commented
2 months ago Thanks very much all for the support and help.
I can confirm that the following patch builds:
--- a/recipes/libwebsockets/all/conanfile.py
+++ b/recipes/libwebsockets/all/conanfile.py
@@ -109,7 +109,7 @@ class LibwebsocketsConan(ConanFile):
"fPIC": True,
"with_libuv": False,
"with_libevent": False,
- "with_zlib": False,
+ "with_zlib": "zlib",
"with_ssl": "openssl",
"with_sqlite3": False,
"with_libmount": False,
@@ -225,7 +225,7 @@ class LibwebsocketsConan(ConanFile):
if self.options.with_ssl == "openssl":
# Cannot add the [>=1.1 <4] range, as it seems openssl3 makes it fail
- self.requires("openssl/1.1.1w", transitive_headers=True)
+ self.requires("openssl/[>=1.1.1w <4]", transitive_headers=True)
elif self.options.with_ssl == "mbedtls":
self.requires("mbedtls/3.5.0")
elif self.options.with_ssl == "wolfssl":I still don't fully understand why the zlib information is not correctly propagated. Conan is correctly bringing zlib as a transitive dependency and linking it as transitive dependency of openssl, but isn't this enough for libwebsockets and it is necessary to explicitly define LWS_WITH_ZLIB cmake var?
jcar87
commented
2 months ago Hi @zzblydia - thanks so much for taking the time to troubleshoot.
In the end, I think it has to do with CHECK_FUNCTION_EXISTS, which relies on CMake's try_compile functionality. The CMAKE_REQUIRED_LIBRARIES need to contain not only OpenSSL, but ZLIB as well, if the former depends on the latter.
These days this is typically handled automatically by CMake, when using targets - try_compile accepts modern imported targets, as well as legacy variables.
What was confusing for us was that there seems to be 3 ways to locate OpenSSL:
LWS_OPENSSL_LIBRARIES and LWS_OPENSSL_INCLUDE_DIRS when invoking CMake. This is unusual, but this is where the issue was; if OpenSSL depends on ZLIB, then ZLIB needs to be passed here too.find_package(OpenSSL). Here pkg-config seems to be redundant, and the build scripts then combine the contents of OPENSSL_LIBRARIES - but if Zlib is correctly listed here (when using OpenSSL 3), then the test passes correctly.In most cases, CMake's legacy dependency variables (xxx_LIBRARIES, etc), can contain a modern CMake target and still work as intended - I would perhaps advice to consider a more "direct" way of locating OpenSSL (with find_package(OpenSSL) first, as that would have prevented this issue altogether.
Hi!
I am trying to build
libwebsockets/4.3.2, with openssl, and exactly the same build withopenssl/1.1.1tbuilds perfectly, but the same build withopenssl/3.1is throwing several of these errors:I couldn't find information in the site or in Github, so I'd like to know if it is possible to use openssl 3.1 (note, same errors happen too with
openssl/3.0.8), and it should work, or this is not supported. Many thanks!