Curl 7.81.0 update fails to detect OpenSSL due to brotli 1.0.9 pkgconfig update

[wrowe]

The current libcurl 7.81.0 update breaks interoperability on OSS builds of OpenSSL 1.1.1m on RHEL.

The nghttp2 and apr-util components are detecting and using the openssl component successfully, this is unique to Curl. Other information here suggests radical breakage of Curl's OpenSSL consumption; https://github.com/envoyproxy/envoy/pull/19407

This may be remedied with changing ./configure flags provided to the linux builds, we will proceed on this plan Monday.

''' mkdir curl-7.81.0 2>/dev/null cd curl-7.81.0 && \ LD_LIBRARY_PATH=/home/build/220107/oss-httpd-build/dst/httpd-2.4.52-220107/lib: \ /home/build/220107/oss-httpd-build/src/curl-7.81.0/configure \ --with-ssl=/home/build/220107/oss-httpd-build/dst/httpd-2.4.52-220107 \ \ --with-brotli=/home/build/220107/oss-httpd-build/dst/httpd-2.4.52-220107 \ --with-nghttp2=/home/build/220107/oss-httpd-build/dst/httpd-2.4.52-220107 \ --without-gnutls \ --without-polarssl \ --without-mbedtls \ --without-cyassl \ --without-nss \ --without-axtls \ --without-libpsl \ --without-libgsasl \ --without-libssh2 \ --without-gssapi \ --without-libidn2 \ --without-librtmp \ --without-zsh-functions-dir \ --disable-ldap \ --disable-ldaps \ --disable-rtsp \ --prefix=/home/build/220107/oss-httpd-build/dst/httpd-2.4.52-220107 && \ cd .. checking whether to enable maintainer-specific portions of Makefiles... no checking whether make supports nested variables... yes checking whether to enable debug build options... no checking whether to enable compiler optimizer... (assumed) yes checking whether to enable strict compiler warnings... no checking whether to enable compiler warnings as errors... no checking whether to enable curl debug memory tracking... no checking whether to enable hiding of library internal symbols... yes checking whether to enable c-ares for DNS lookups... no checking whether to disable dependency on -lrt... (assumed no) checking whether to enable ECH support... no checking for path separator... : checking for sed... /usr/bin/sed checking for grep... /usr/bin/grep checking for egrep... /usr/bin/grep -E checking for ar... /usr/bin/ar checking for a BSD-compatible install... /usr/bin/install -c checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether the compiler supports GNU C... yes checking whether gcc accepts -g... yes checking for gcc option to enable C11 features... -std=gnu11 checking whether gcc -std=gnu11 understands -c and -o together... yes checking how to run the C preprocessor... gcc -std=gnu11 -E checking for a sed that does not truncate output... (cached) /usr/bin/sed checking for code coverage support... no checking whether build environment is sane... yes checking for a race-free mkdir -p... /usr/bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking whether make supports the include directive... yes (GNU style) checking dependency style of gcc -std=gnu11... gcc3 checking curl version... 7.81.0 checking build system type... x86_64-pc-linux-gnu checking host system type... x86_64-pc-linux-gnu checking for grep that handles long lines and -e... (cached) /usr/bin/grep checking for egrep... /usr/bin/grep -E checking if OS is AIX (to define _ALL_SOURCE)... no checking if _THREAD_SAFE is already defined... no checking if _THREAD_SAFE is actually needed... no checking if _THREAD_SAFE is onwards defined... no checking if _REENTRANT is already defined... no checking if _REENTRANT is actually needed... no checking if _REENTRANT is onwards defined... no checking for special C compiler options needed for large files... no checking for _FILE_OFFSET_BITS value needed for large files... no checking how to print strings... printf checking for a sed that does not truncate output... (cached) /usr/bin/sed checking for fgrep... /usr/bin/grep -F checking for ld used by gcc -std=gnu11... /usr/bin/ld checking if the linker (/usr/bin/ld) is GNU ld... yes checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B checking the name lister (/usr/bin/nm -B) interface... BSD nm checking whether ln -s works... yes checking the maximum length of command line arguments... 1572864 checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop checking for /usr/bin/ld option to reload object files... -r checking for objdump... objdump checking how to recognize dependent libraries... pass_all checking for dlltool... no checking how to associate runtime and link libraries... printf %s\n checking for archiver @FILE support... @ checking for strip... strip checking for ranlib... ranlib checking command to parse /usr/bin/nm -B output from gcc -std=gnu11 object... ok checking for sysroot... no checking for a working dd... /usr/bin/dd checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 checking for mt... no checking if : is a manifest tool... no checking for stdio.h... yes checking for stdlib.h... yes checking for string.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for strings.h... yes checking for sys/stat.h... yes checking for sys/types.h... yes checking for unistd.h... yes checking for dlfcn.h... yes checking for objdir... .libs checking if gcc -std=gnu11 supports -fno-rtti -fno-exceptions... no checking for gcc -std=gnu11 option to produce PIC... -fPIC -DPIC checking if gcc -std=gnu11 PIC flag -fPIC -DPIC works... yes checking if gcc -std=gnu11 static flag -static works... no checking if gcc -std=gnu11 supports -c -o file.o... yes checking if gcc -std=gnu11 supports -c -o file.o... (cached) yes checking whether the gcc -std=gnu11 linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking whether -lc should be explicitly linked in... no checking dynamic linker characteristics... GNU/Linux ld.so checking how to hardcode library paths into programs... immediate checking whether stripping libraries is possible... yes checking if libtool supports shared libraries... yes checking whether to build shared libraries... yes checking whether to build static libraries... yes checking whether to build shared libraries with -version-info... yes checking whether to build shared libraries with -no-undefined... no checking whether to build shared libraries with -mimpure-text... no checking whether to build shared libraries with PIC... yes checking whether to build static libraries with PIC... yes checking whether to build shared libraries only... no checking whether to build static libraries only... no checking for inline... inline checking if cpp -P is needed... no checking if compiler is DEC/Compaq/HP C... no checking if compiler is HP-UX C... no checking if compiler is IBM C... no checking if compiler is Intel C... no checking if compiler is clang... no checking if compiler is GNU C... yes checking if compiler is LCC... no checking if compiler is SGI MIPSpro C... no checking if compiler is SGI MIPS C... no checking if compiler is SunPro C... no checking if compiler is Tiny C... no checking if compiler is Watcom C... no checking if compiler accepts some basic options... yes configure: compiler options added: -Werror-implicit-function-declaration checking if compiler optimizer assumed setting might be used... yes checking if compiler accepts optimizer enabling options... yes configure: compiler options added: -O2 checking if compiler accepts strict warning options... yes configure: compiler options added: -Wno-system-headers checking if compiler halts on compilation errors... yes checking if compiler halts on negative sized arrays... yes checking if compiler halts on function prototype mismatch... yes checking if compiler supports hiding library internal symbols... yes checking for windows.h... no checking whether build target is a native Windows one... no checking whether build target supports WIN32 file API... no checking whether build target supports WIN32 crypto API... no checking for good-to-use Darwin CFLAGS... no checking whether to link macOS CoreFoundation and SystemConfiguration framework... no checking to see if the compiler supports __builtin_available()... no checking whether to support http... yes checking whether to support ftp... yes checking whether to support file... yes checking whether to support ldap... no checking whether to support ldaps... no checking whether to support rtsp... no checking whether to support proxies... yes checking whether to support dict... yes checking whether to support telnet... yes checking whether to support tftp... yes checking whether to support pop3... yes checking whether to support imap... yes checking whether to support smb... yes checking whether to support smtp... yes checking whether to support gopher... yes checking whether to support mqtt... no checking whether to provide built-in manual... yes checking whether to enable generation of C code... yes checking whether to use libgcc... no checking if X/Open network library is required... no checking for gethostbyname... yes checking for windows.h... (cached) no checking for winsock2.h... (cached) no checking for proto/bsdsocket.h... no checking for connect in libraries... yes checking for sys/types.h... (cached) yes checking for sys/time.h... yes checking for monotonic clock_gettime... yes checking for clock_gettime in libraries... no additional lib required checking if monotonic clock_gettime works... yes checking for pkg-config... /usr/bin/pkg-config checking for zlib options with pkg-config... found checking for zlib.h... yes configure: found both libz and libz.h header checking for BrotliDecoderDecompress in -lbrotlidec... no checking for brotli/decode.h... yes configure: Added /home/build/220107/oss-httpd-build/dst/httpd-2.4.52-220107/lib to CURL_LIBRARY_PATH checking for ZSTD_createDStream in -lzstd... no checking for zstd.h... no checking whether to enable IPv6... no checking if argv can be written to... yes checking if GSS-API support is requested... no checking whether to enable Windows native SSL/TLS... no checking whether to enable Secure Transport... no checking whether to enable Amiga native SSL/TLS (AmiSSL)... no configure: PKG_CONFIG_LIBDIR will be set to "/home/build/220107/oss-httpd-build/dst/httpd-2.4.52-220107/lib/pkgconfig" checking for pkg-config... (cached) /usr/bin/pkg-config checking for openssl options with pkg-config... found configure: pkg-config: SSL_LIBS: "-lssl -lcrypto " configure: pkg-config: SSL_LDFLAGS: "-L/home/build/220107/oss-httpd-build/dst/httpd-2.4.52-220107/lib " configure: pkg-config: SSL_CPPFLAGS: "-I/home/build/220107/oss-httpd-build/dst/httpd-2.4.52-220107/include " checking for HMAC_Update in -lcrypto... no checking for HMAC_Init_ex in -lcrypto... no checking OpenSSL linking with -ldl... no checking OpenSSL linking with -ldl and -lpthread... no configure: OPT_OPENSSL: /home/build/220107/oss-httpd-build/dst/httpd-2.4.52-220107 configure: OPENSSL_ENABLED: configure: error: --with-openssl was given but OpenSSL could not be detected make: *** [curl-7.81.0/Makefile] Error 1 '''

[wrowe]

Reopening until this is duly reported to the brotli and curl communities

[wrowe]

In summary, -lcrypto -lssl didn't work on rhel 7 with brotli 1.0.9 and openssl 1.1.1m because -lbrotlienc couldn't resolve -lbrotlicommon.

Looked like an OpenSSL failure, but it was curl's use of brotli pkgconfig that actually breaks in 7.81.0 with autoconf. cmake appears to be unaffected.

[wrowe]

Retested against current brotli / openssl / curl maintenance branches, and the error has been resolved upstream;

checking for BrotliDecoderDecompress in -lbrotlidec... yes
checking brotli/decode.h usability... yes
checking brotli/decode.h presence... yes
checking for brotli/decode.h... yes
configure: Added /mnt/c/Users/wrowe/dev/oss-httpd-build/dst/httpd-2.4.x-1904212/lib to CURL_LIBRARY_PATH
checking for ZSTD_createDStream in -lzstd... no
checking zstd.h usability... no
checking zstd.h presence... no
checking for zstd.h... no
checking whether to enable IPv6... yes
checking if struct sockaddr_in6 has sin6_scope_id member... yes
checking if argv can be written to... yes
checking if GSS-API support is requested... no
checking whether to enable Windows native SSL/TLS... no
checking whether to enable Secure Transport... no
checking whether to enable Amiga native SSL/TLS (AmiSSL v5)... no
configure: PKG_CONFIG_LIBDIR will be set to "/mnt/c/Users/wrowe/dev/oss-httpd-build/dst/httpd-2.4.x-1904212/lib/pkgconfig"
checking for pkg-config... /usr/bin/pkg-config
checking for openssl options with pkg-config... found
configure: pkg-config: SSL_LIBS: "-lssl -lcrypto"
configure: pkg-config: SSL_LDFLAGS: "-L/mnt/c/Users/wrowe/dev/oss-httpd-build/dst/httpd-2.4.x-1904212/lib"
configure: pkg-config: SSL_CPPFLAGS: "-I/mnt/c/Users/wrowe/dev/oss-httpd-build/dst/httpd-2.4.x-1904212/include"
checking for HMAC_Update in -lcrypto... yes
checking for SSL_connect in -lssl... yes
checking openssl/x509.h usability... yes
checking openssl/x509.h presence... yes
checking for openssl/x509.h... yes
checking openssl/rsa.h usability... yes
checking openssl/rsa.h presence... yes
checking for openssl/rsa.h... yes
checking openssl/crypto.h usability... yes
checking openssl/crypto.h presence... yes
checking for openssl/crypto.h... yes
checking openssl/pem.h usability... yes
checking openssl/pem.h presence... yes
checking for openssl/pem.h... yes
checking openssl/ssl.h usability... yes
checking openssl/ssl.h presence... yes
checking for openssl/ssl.h... yes
checking openssl/err.h usability... yes
checking openssl/err.h presence... yes
checking for openssl/err.h... yes
checking for RAND_egd... no
checking for BoringSSL... no
checking for libressl... no
checking for OpenSSL >= v3... yes
configure: Added /mnt/c/Users/wrowe/dev/oss-httpd-build/dst/httpd-2.4.x-1904212/lib to CURL_LIBRARY_PATH

[wrowe]

Reconfirmed against current release build.