curl / curl

A command line tool and library for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET, TFTP, WS and WSS. libcurl offers a myriad of powerful features
https://curl.se/
Other
35.44k stars 6.37k forks source link

Regression: curl 8.8.0 links with openssl despite --without-openssl #13747

Closed kpcyrd closed 4 months ago

kpcyrd commented 4 months ago

I did this

I built a curl binary with the rustls backend, and openssl explicitly disabled:

  local _configure_options=(
    --prefix='/usr'
    --mandir='/usr/share/man'
    --disable-ldap
    --disable-ldaps
    --disable-manual
    --enable-ipv6
    --enable-threaded-resolver
    --with-gssapi
    --with-libssh2
    --with-random='/dev/urandom'
    --with-ca-bundle='/etc/ssl/certs/ca-certificates.crt'
  )

  mkdir build-curl

  # build curl
  cd "${srcdir}"/build-curl

  "${srcdir}/curl"/configure \
    "${_configure_options[@]}" \
    --with-rustls \
    --without-openssl \
    --disable-shared
  sed -i -e 's/ -shared / -Wl,-O1,--as-needed\0/g' libtool
  make

It prints this summary at the end of configure:

configure: Configured to build curl/libcurl:

  Host setup:       x86_64-pc-linux-gnu
  Install prefix:   /usr
  Compiler:         gcc
   CFLAGS:          -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -ffile-prefix-map=/build/curl-rustls/src=/usr/src/debug/curl-rustls -flto=auto -Werror-implicit-function-declaration -Wno-system-headers
   CPPFLAGS:        
   LDFLAGS:         
   LIBS:            -lnghttp2 -lidn2 -lssh2 -lssh2 -lssl -lcrypto -lpsl -lrustls -lgssapi_krb5 -lzstd -lbrotlidec -lz

  curl version:     8.8.0
  SSL:              enabled (rustls)
  SSH:              enabled (libSSH2)
  zlib:             enabled
  brotli:           enabled (libbrotlidec)
  zstd:             enabled (libzstd)
  GSS-API:          enabled (MIT Kerberos/Heimdal)
  GSASL:            no      (libgsasl not found)
  TLS-SRP:          no      (--enable-tls-srp)
  resolver:         POSIX threaded
  IPv6:             enabled
  Unix sockets:     enabled
  IDN:              enabled (libidn2)
  Build docs:       enabled (--disable-docs)
  Build libcurl:    Shared=no, Static=yes
  Built-in manual:  no      (--enable-manual)
  --libcurl option: enabled (--disable-libcurl-option)
  Verbose errors:   enabled (--disable-verbose)
  Code coverage:    disabled
  SSPI:             no      (--enable-sspi)
  ca cert bundle:   /etc/ssl/certs/ca-certificates.crt
  ca cert path:     no
  ca fallback:      no
  LDAP:             no      (--enable-ldap / --with-ldap-lib / --with-lber-lib)
  LDAPS:            no      (--enable-ldaps)
  RTSP:             enabled
  RTMP:             no      (--with-librtmp)
  PSL:              enabled
  Alt-svc:          enabled (--disable-alt-svc)
  Headers API:      enabled (--disable-headers-api)
  HSTS:             enabled (--disable-hsts)
  HTTP1:            enabled (internal)
  HTTP2:            enabled (nghttp2)
  HTTP3:            no      (--with-ngtcp2 --with-nghttp3, --with-quiche, --with-openssl-quic, --with-msh3)
  ECH:              no      (--enable-ech)
  WebSockets:       no      (--enable-websockets)
  Protocols:        DICT FILE FTP FTPS GOPHER GOPHERS HTTP HTTPS IMAP IMAPS IPFS IPNS MQTT POP3 POP3S RTSP SCP SFTP SMTP SMTPS TELNET TFTP
  Features:         AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile PSL SPNEGO SSL UnixSockets alt-svc brotli libz threadsafe zstd

  WARNING:  rustls enabled but marked EXPERIMENTAL. Use with caution!

I expected the following

Arch Linux dev tooling flagged the binary as over-linked:

curl-rustls W: Unused shared library '/usr/lib/libssl.so.3' by file ('usr/lib/curl-rustls/curl')
curl-rustls W: Unused shared library '/usr/lib/libcrypto.so.3' by file ('usr/lib/curl-rustls/curl')
curl-rustls W: Dependency openssl detected and implicitly satisfied (libraries ['usr/lib/libssl.so.3', 'usr/lib/libcrypto.so.3'] needed in files ['usr/lib/curl-rustls/curl'])

and indeed, it wants the linker to load libssl and libcrypto into the process:

% readelf -d /usr/bin/curl-rustls | rg NEED
 0x0000000000000001 (NEEDED)             Shared library: [libnghttp2.so.14]
 0x0000000000000001 (NEEDED)             Shared library: [libidn2.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libssh2.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libssl.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libcrypto.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libpsl.so.5]
 0x0000000000000001 (NEEDED)             Shared library: [librustls.so.0.13.0]
 0x0000000000000001 (NEEDED)             Shared library: [libgssapi_krb5.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libzstd.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libbrotlidec.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libz.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000006ffffffe (VERNEED)            0x3bb8
 0x000000006fffffff (VERNEEDNUM)         3

This looks like another autotools issue.

curl/libcurl version

% curl-rustls --version
curl 8.8.0 (x86_64-pc-linux-gnu) libcurl/8.8.0 rustls-ffi/0.13.0/rustls/0.23.4 zlib/1.3.1 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.7 libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.62.0
Release-Date: 2024-05-22
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz PSL SPNEGO SSL threadsafe UnixSockets zstd

operating system

Arch Linux

bagder commented 4 months ago

A good first trick is to check your config.log where it gets introduced into the LIBS/CFLAGS.

What is your libssh2 using for example?

icing commented 4 months ago

When I configure rustls and libssh2, the -lssl -lcrypto do not get added.

bagder commented 4 months ago

A common pattern when things like this happened in the past, it has been bad pkg-config files from some package curl gets the options from during configure.

kpcyrd commented 4 months ago

libssh2 depends on openssl itself, but I'm explicitly using readelf instead of ldd. If it would be a transitive dependency it wouldn't show up, but the curl build itself is passing -lssl -lcrypto to the linker (as can be seen in $LIBS).

``` This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. It was created by curl configure -, which was generated by GNU Autoconf 2.72. Invocation command line was $ /build/curl-rustls/src/curl/configure --prefix=/usr --mandir=/usr/share/man --disable-ldap --disable-ldaps --disable-manual --enable-ipv6 --enable-threaded-resolver --with-gssapi --with-libssh2 --with-random=/dev/urandom --with-ca-bundle=/etc/ssl/certs/ca-certificates.crt --with-rustls --without-openssl --disable-shared ## --------- ## ## Platform. ## ## --------- ## hostname = arch-nspawn-3346228 uname -m = x86_64 uname -r = 6.9.1-arch1-1 uname -s = Linux uname -v = #1 SMP PREEMPT_DYNAMIC Fri, 17 May 2024 16:56:38 +0000 [...] ## ----------------- ## ## Output variables. ## ## ----------------- ## ACLOCAL='${SHELL} '\''/build/curl-rustls/src/curl/missing'\'' aclocal-1.16' AMDEPBACKSLASH='\' AMDEP_FALSE='#' AMDEP_TRUE='' AMTAR='$${TAR-tar}' AM_BACKSLASH='\' AM_DEFAULT_V='$(AM_DEFAULT_VERBOSITY)' AM_DEFAULT_VERBOSITY='0' AM_V='$(V)' APACHECTL='' APXS='' AR='/usr/bin/ar' AR_FLAGS='cr' AS='as' AUTOCONF='${SHELL} '\''/build/curl-rustls/src/curl/missing'\'' autoconf' AUTOHEADER='${SHELL} '\''/build/curl-rustls/src/curl/missing'\'' autoheader' AUTOMAKE='${SHELL} '\''/build/curl-rustls/src/curl/missing'\'' automake-1.16' AWK='gawk' BLANK_AT_MAKETIME='' BUILD_DOCS_FALSE='#' BUILD_DOCS_TRUE='' BUILD_LIBHOSTNAME_FALSE='' BUILD_LIBHOSTNAME_TRUE='#' BUILD_STUB_GSS_FALSE='#' BUILD_STUB_GSS_TRUE='' BUILD_UNITTESTS_FALSE='' BUILD_UNITTESTS_TRUE='#' CADDY='/usr/bin/caddy' CC='gcc' CCDEPMODE='depmode=gcc3' CFLAGS='-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -ffile-prefix-map=/build/curl-rustls/src=/usr/src/debug/curl-rustls -flto=auto -Werror-implicit-function-declaration -Wno-system-headers' CFLAG_CURL_SYMBOL_HIDING='-fvisibility=hidden' CONFIGURE_OPTIONS='" '\''--prefix=/usr'\'' '\''--mandir=/usr/share/man'\'' '\''--disable-ldap'\'' '\''--disable-ldaps'\'' '\''--disable-manual'\'' '\''--enable-ipv6'\'' '\''--enable-threaded-resolver'\'' '\''--with-gssapi'\'' '\''--with-libssh2'\'' '\''--with-random=/dev/urandom'\'' '\''--with-ca-bundle=/etc/ssl/certs/ca-certificates.crt'\'' '\''--with-rustls'\'' '\''--without-openssl'\'' '\''--disable-shared'\'' '\''CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -ffile-prefix-map=/build/curl-rustls/src=/usr/src/debug/curl-rustls -flto=auto'\'' '\''LDFLAGS=-Wl,-O1 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,-z,pack-relative-relocs -flto=auto'\''"' CPP='gcc -E' CPPFLAGS='' CPPFLAG_CURL_STATICLIB='-DCURL_STATICLIB' CROSSCOMPILING_FALSE='' CROSSCOMPILING_TRUE='#' CSCOPE='cscope' CTAGS='ctags' CURLDEBUG_FALSE='' CURLDEBUG_TRUE='#' CURLVERSION='8.8.0' CURL_CA_BUNDLE='"/etc/ssl/certs/ca-certificates.crt"' CURL_CFLAG_EXTRAS='' CURL_DISABLE_DICT='' CURL_DISABLE_FILE='' CURL_DISABLE_FTP='' CURL_DISABLE_GOPHER='' CURL_DISABLE_HTTP='' CURL_DISABLE_IMAP='' CURL_DISABLE_LDAP='1' CURL_DISABLE_LDAPS='1' CURL_DISABLE_MQTT='' CURL_DISABLE_POP3='' CURL_DISABLE_PROXY='' CURL_DISABLE_RTSP='' CURL_DISABLE_SMB='' CURL_DISABLE_SMTP='' CURL_DISABLE_TELNET='' CURL_DISABLE_TFTP='' CURL_LT_SHLIB_USE_MIMPURE_TEXT_FALSE='' CURL_LT_SHLIB_USE_MIMPURE_TEXT_TRUE='#' CURL_LT_SHLIB_USE_NO_UNDEFINED_FALSE='' CURL_LT_SHLIB_USE_NO_UNDEFINED_TRUE='#' CURL_LT_SHLIB_USE_VERSIONED_SYMBOLS_FALSE='' CURL_LT_SHLIB_USE_VERSIONED_SYMBOLS_TRUE='#' CURL_LT_SHLIB_USE_VERSION_INFO_FALSE='#' CURL_LT_SHLIB_USE_VERSION_INFO_TRUE='' CURL_LT_SHLIB_VERSIONED_FLAVOUR='' CURL_NETWORK_AND_TIME_LIBS='' CURL_NETWORK_LIBS='' CURL_WITH_MULTI_SSL='' CYGPATH_W='echo' DEFAULT_SSL_BACKEND='no' DEFS='-DHAVE_CONFIG_H' DEPDIR='.deps' DLLTOOL='false' DOING_CURL_SYMBOL_HIDING_FALSE='#' DOING_CURL_SYMBOL_HIDING_TRUE='' DOING_NATIVE_WINDOWS_FALSE='' DOING_NATIVE_WINDOWS_TRUE='#' DSYMUTIL='' DUMPBIN='' ECHO_C='' ECHO_N='-n' ECHO_T='' EGREP='/usr/bin/grep -E' ENABLE_SHARED='no' ENABLE_STATIC='yes' ETAGS='etags' EXEEXT='' FGREP='/usr/bin/grep -F' FILECMD=':' FISH_FUNCTIONS_DIR='' GCOV='' GREP='/usr/bin/grep' HAVE_BROTLI='1' HAVE_GNUTLS_SRP='' HAVE_LDAP_SSL='' HAVE_LIBZ='1' HAVE_LIBZ_FALSE='#' HAVE_LIBZ_TRUE='' HAVE_OPENSSL_QUIC='' HAVE_OPENSSL_SRP='' HAVE_PROTO_BSDSOCKET_H='' HAVE_WINDRES_FALSE='' HAVE_WINDRES_TRUE='#' HAVE_ZSTD='1' HTTPD='' HTTPD_NGHTTPX='' IDN_ENABLED='1' INSTALL_DATA='${INSTALL} -m 644' INSTALL_PROGRAM='${INSTALL}' INSTALL_SCRIPT='${INSTALL}' INSTALL_STRIP_PROGRAM='$(install_sh) -c -s' IPV6_ENABLED='1' LCOV='' LD='/usr/bin/ld' LDFLAGS='' LIBCURL_LIBS='-lnghttp2 -lidn2 -lssh2 -lssh2 -lssl -lcrypto -lpsl -lrustls -lgssapi_krb5 -lzstd -lbrotlidec -lz' LIBCURL_NO_SHARED='-lnghttp2 -lidn2 -lssh2 -lssh2 -lssl -lcrypto -lpsl -lrustls -lgssapi_krb5 -lzstd -lbrotlidec -lz ' LIBOBJS='' LIBS='-lnghttp2 -lidn2 -lssh2 -lssh2 -lssl -lcrypto -lpsl -lrustls -lgssapi_krb5 -lzstd -lbrotlidec -lz' LIBTOOL='$(SHELL) $(top_builddir)/libtool' LIPO='' LN_S='ln -s' LTLIBOBJS='' LT_SYS_LIBRARY_PATH='' MAINT='#' MAINTAINER_MODE_FALSE='' MAINTAINER_MODE_TRUE='#' MAKEINFO='${SHELL} '\''/build/curl-rustls/src/curl/missing'\'' makeinfo' MANIFEST_TOOL=':' MKDIR_P='/usr/bin/mkdir -p' NM='/usr/bin/nm -B' NMEDIT='' OBJDUMP='objdump' OBJEXT='o' OTOOL64='' OTOOL='' PACKAGE='curl' PACKAGE_BUGREPORT='a suitable curl mailing list: https://curl.se/mail/' PACKAGE_NAME='curl' PACKAGE_STRING='curl -' PACKAGE_TARNAME='curl' PACKAGE_URL='' PACKAGE_VERSION='-' PATH_SEPARATOR=':' PERL='/usr/bin/perl' PKGADD_NAME='curl - a client that groks URLs' PKGADD_PKG='HAXXcurl' PKGADD_VENDOR='curl.se' PKGCONFIG='/usr/bin/pkg-config' RANDOM_FILE='' RANLIB='ranlib' RC='' REQUIRE_LIB_DEPS='yes' SED='/usr/bin/sed' SET_MAKE='' SHELL='/bin/sh' SSL_BACKENDS='rustls' SSL_ENABLED='1' SSL_LIBS='-lrustls' STRIP='strip' SUPPORT_FEATURES='AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile PSL SPNEGO SSL UnixSockets alt-svc brotli libz threadsafe zstd' SUPPORT_PROTOCOLS='DICT FILE FTP FTPS GOPHER GOPHERS HTTP HTTPS IMAP IMAPS IPFS IPNS MQTT POP3 POP3S RTSP SCP SFTP SMTP SMTPS TELNET TFTP' TEST_NGHTTPX='nghttpx' USE_ARES='' USE_BEARSSL='' USE_CPPFLAG_CURL_STATICLIB_FALSE='#' USE_CPPFLAG_CURL_STATICLIB_TRUE='' USE_EXPLICIT_LIB_DEPS_FALSE='#' USE_EXPLICIT_LIB_DEPS_TRUE='' USE_FISH_COMPLETION_FALSE='' USE_FISH_COMPLETION_TRUE='#' USE_GNUTLS='' USE_GSASL_FALSE='' USE_GSASL_TRUE='#' USE_HYPER='' USE_LIBPSL='1' USE_LIBPSL_FALSE='#' USE_LIBPSL_TRUE='' USE_LIBRTMP='' USE_LIBSSH2='1' USE_LIBSSH='' USE_MANUAL_FALSE='' USE_MANUAL_TRUE='#' USE_MBEDTLS='' USE_MSH3='' USE_NGHTTP2='1' USE_NGHTTP3='' USE_NGTCP2='' USE_NGTCP2_CRYPTO_BORINGSSL='' USE_NGTCP2_CRYPTO_GNUTLS='' USE_NGTCP2_CRYPTO_QUICTLS='' USE_NGTCP2_CRYPTO_WOLFSSL='' USE_NGTCP2_H3='' USE_OPENLDAP='' USE_OPENSSL_H3='' USE_OPENSSL_QUIC='' USE_QUICHE='' USE_RUSTLS='yes' USE_SCHANNEL='' USE_SECTRANSP='' USE_UNIX_SOCKETS='1' USE_WIN32_CRYPTO='' USE_WIN32_LARGE_FILES='' USE_WIN32_SMALL_FILES='' USE_WINDOWS_SSPI='' USE_WOLFSSH='' USE_WOLFSSL='' USE_ZSH_COMPLETION_FALSE='' USE_ZSH_COMPLETION_TRUE='#' VERSION='-' VERSIONNUM='080800' VSFTPD='/usr/sbin/vsftpd' ZLIB_LIBS='-lz' ZSH_FUNCTIONS_DIR='' ac_ct_AR='' ac_ct_CC='gcc' ac_ct_DUMPBIN='' am__EXEEXT_FALSE='' am__EXEEXT_TRUE='#' am__fastdepCC_FALSE='#' am__fastdepCC_TRUE='' am__include='include' am__isrc=' -I$(srcdir)' am__leading_dot='.' am__nodep='_no' am__quote='' am__tar='$${TAR-tar} chof - "$$tardir"' am__untar='$${TAR-tar} xf -' bindir='${exec_prefix}/bin' build='x86_64-pc-linux-gnu' build_alias='' build_cpu='x86_64' build_os='linux-gnu' build_vendor='pc' datadir='${datarootdir}' datarootdir='${prefix}/share' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' dvidir='${docdir}' exec_prefix='${prefix}' host='x86_64-pc-linux-gnu' host_alias='' host_cpu='x86_64' host_os='linux-gnu' host_vendor='pc' htmldir='${docdir}' includedir='${prefix}/include' infodir='${datarootdir}/info' install_sh='${SHELL} /build/curl-rustls/src/curl/install-sh' libdir='${exec_prefix}/lib' libexecdir='${exec_prefix}/libexec' libext='a' localedir='${datarootdir}/locale' localstatedir='${prefix}/var' mandir='/usr/share/man' mkdir_p='$(MKDIR_P)' oldincludedir='/usr/include' pdfdir='${docdir}' prefix='/usr' program_transform_name='s,x,x,' psdir='${docdir}' runstatedir='${localstatedir}/run' sbindir='${exec_prefix}/sbin' sharedstatedir='${prefix}/com' sysconfdir='${prefix}/etc' target_alias='' ## ----------- ## ## confdefs.h. ## ## ----------- ## /* confdefs.h */ #define PACKAGE_NAME "curl" #define PACKAGE_TARNAME "curl" #define PACKAGE_VERSION "-" #define PACKAGE_STRING "curl -" #define PACKAGE_BUGREPORT "a suitable curl mailing list: https://curl.se/mail/" #define PACKAGE_URL "" #define HAVE_STDIO_H 1 #define HAVE_STDLIB_H 1 #define HAVE_STRING_H 1 #define HAVE_INTTYPES_H 1 #define HAVE_STDINT_H 1 #define HAVE_STRINGS_H 1 #define HAVE_SYS_STAT_H 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_UNISTD_H 1 #define STDC_HEADERS 1 #define HAVE_STDATOMIC_H 1 #define HAVE_ATOMIC 1 #define PACKAGE "curl" #define VERSION "-" #define OS "x86_64-pc-linux-gnu" #define HAVE_DLFCN_H 1 #define LT_OBJDIR ".libs/" #define CURL_DISABLE_LDAP 1 #define CURL_DISABLE_LDAPS 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_SYS_TIME_H 1 #define HAVE_CLOCK_GETTIME_MONOTONIC 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_SYS_TIME_H 1 #define HAVE_CLOCK_GETTIME_MONOTONIC_RAW 1 #define HAVE_LIBZ 1 #define HAVE_LIBBROTLIDEC 1 #define HAVE_BROTLI_DECODE_H 1 #define HAVE_BROTLI 1 #define HAVE_LIBZSTD 1 #define HAVE_ZSTD_H 1 #define HAVE_ZSTD 1 #define USE_IPV6 1 #define HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID 1 #define HAVE_WRITABLE_ARGV 1 #define HAVE_GSSAPI_GSSAPI_H 1 #define HAVE_GSSAPI_GSSAPI_GENERIC_H 1 #define HAVE_GSSAPI_GSSAPI_KRB5_H 1 #define HAVE_GSSAPI 1 #define USE_RUSTLS 1 #define CURL_CA_BUNDLE "/etc/ssl/certs/ca-certificates.crt" #define HAVE_LIBPSL_H 1 #define USE_LIBPSL 1 #define HAVE_LIBSSH2 1 #define USE_LIBSSH2 1 #define HAVE_IDN2_H 1 #define HAVE_LIBIDN2 1 #define HAVE_NGHTTP2_NGHTTP2_H 1 #define USE_NGHTTP2 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_SYS_TIME_H 1 #define HAVE_SYS_SELECT_H 1 #define HAVE_SYS_SOCKET_H 1 #define HAVE_SYS_IOCTL_H 1 #define HAVE_UNISTD_H 1 #define HAVE_STDLIB_H 1 #define HAVE_ARPA_INET_H 1 #define HAVE_NET_IF_H 1 #define HAVE_NETINET_IN_H 1 #define HAVE_SYS_UN_H 1 #define HAVE_LINUX_TCP_H 1 #define HAVE_NETINET_TCP_H 1 #define HAVE_NETINET_UDP_H 1 #define HAVE_NETDB_H 1 #define HAVE_SYS_STAT_H 1 #define HAVE_SYS_PARAM_H 1 #define HAVE_TERMIOS_H 1 #define HAVE_TERMIO_H 1 #define HAVE_FCNTL_H 1 #define HAVE_PWD_H 1 #define HAVE_UTIME_H 1 #define HAVE_SYS_POLL_H 1 #define HAVE_POLL_H 1 #define HAVE_SYS_RESOURCE_H 1 #define HAVE_LIBGEN_H 1 #define HAVE_LOCALE_H 1 #define HAVE_STDBOOL_H 1 #define HAVE_SYS_WAIT_H 1 #define HAVE_SETJMP_H 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_SYS_TIME_H 1 #define HAVE_SYS_SOCKET_H 1 #define HAVE_STRUCT_TIMEVAL 1 #define SIZEOF_SIZE_T 8 #define SIZEOF_LONG 8 #define SIZEOF_INT 4 #define SIZEOF_TIME_T 8 #define SIZEOF_OFF_T 8 #define SIZEOF_CURL_OFF_T 8 #define SIZEOF_CURL_SOCKET_T 4 #define HAVE_LONGLONG 1 #define HAVE_BOOL_T 1 #define CURL_SA_FAMILY_T sa_family_t #define HAVE_SUSECONDS_T 1 #define HAVE_STRUCT_SOCKADDR_STORAGE 1 #define HAVE_SYS_SELECT_H 1 #define HAVE_SYS_SOCKET_H 1 #define HAVE_SELECT 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_SYS_SOCKET_H 1 #define HAVE_RECV 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_SYS_SOCKET_H 1 #define HAVE_SEND 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_SYS_SOCKET_H 1 #define HAVE_MSG_NOSIGNAL 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_UNISTD_H 1 #define HAVE_ALARM 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_STRINGS_H 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_LIBGEN_H 1 #define HAVE_BASENAME 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_SYS_SOCKET_H 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_UNISTD_H 1 #define HAVE_FCNTL_H 1 #define HAVE_FCNTL 1 #define HAVE_FCNTL_O_NONBLOCK 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_NETDB_H 1 #define HAVE_FREEADDRINFO 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_SYS_XATTR_H 1 #define HAVE_FSETXATTR 1 #define HAVE_FSETXATTR_5 1 #define HAVE_FTRUNCATE 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_GETADDRINFO 1 #define HAVE_GETADDRINFO_THREADSAFE 1 #define HAVE_GETHOSTBYNAME 1 #define HAVE_GETHOSTBYNAME_R 1 #define HAVE_GETHOSTBYNAME_R_6 1 #define GETHOSTNAME_TYPE_ARG2 size_t #define HAVE_GETHOSTNAME 1 #define HAVE_GETPEERNAME 1 #define HAVE_GETSOCKNAME 1 #define HAVE_NET_IF_H 1 #define HAVE_IF_NAMETOINDEX 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_SYS_SOCKET_H 1 #define HAVE_NETINET_IN_H 1 #define HAVE_IFADDRS_H 1 #define HAVE_GETIFADDRS 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_SYS_TIME_H 1 #define HAVE_GMTIME_R 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_SYS_SOCKET_H 1 #define HAVE_NETINET_IN_H 1 #define HAVE_ARPA_INET_H 1 #define HAVE_INET_NTOP 1 #define HAVE_INET_PTON 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_UNISTD_H 1 #define HAVE_SYS_SOCKET_H 1 #define HAVE_SYS_IOCTL_H 1 #define HAVE_IOCTL 1 #define HAVE_IOCTL_FIONBIO 1 #define HAVE_IOCTL_SIOCGIFADDR 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_POLL_H 1 #define HAVE_SYS_POLL_H 1 #define HAVE_POLL_FINE 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_SIGACTION 1 #define HAVE_SIGINTERRUPT 1 #define HAVE_SIGNAL 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_SIGSETJMP 1 #define HAVE_SOCKET 1 #define HAVE_SOCKETPAIR 1 #define HAVE_STRCASECMP 1 #define HAVE_STRDUP 1 #define HAVE_STRERROR_R 1 #define HAVE_POSIX_STRERROR_R 1 #define HAVE_STRTOK_R 1 #define HAVE_STRTOLL 1 #define HAVE_DECL_GETPWUID_R 1 #define HAVE_ARC4RANDOM 1 #define HAVE_FNMATCH 1 #define HAVE_FSEEKO 1 #define HAVE_GETEUID 1 #define HAVE_GETPPID 1 #define HAVE_GETPWUID 1 #define HAVE_GETPWUID_R 1 #define HAVE_GETRLIMIT 1 #define HAVE_GETTIMEOFDAY 1 #define HAVE_IF_NAMETOINDEX 1 #define HAVE_PIPE 1 #define HAVE_SCHED_YIELD 1 #define HAVE_SENDMSG 1 #define HAVE_SETLOCALE 1 #define HAVE_SETRLIMIT 1 #define HAVE_SNPRINTF 1 #define HAVE_UTIME 1 #define HAVE_UTIMES 1 #define HAVE_DECL_FSEEKO 1 #define HAVE_PTHREAD_H 1 #define USE_THREADS_POSIX 1 #define HAVE_DIRENT_H 1 #define HAVE_OPENDIR 1 #define USE_UNIX_SOCKETS 1 #define CURL_EXTERN_SYMBOL __attribute__ ((__visibility__ ("default"))) configure: exit 0 ```

full config.log

bagder commented 4 months ago

It looks like -lssl is first used by configure exactly after it got the data from pkg-config for libssh2...

kpcyrd commented 4 months ago

I have this pkgconfig file on my system, but I didn't have the time yet to check if Libs: was recently changed (but it seems wrong to have -lssl and -lcrypto in there).

% cat /usr/lib/pkgconfig/libssh2.pc
###########################################################################
# libssh2 installation details
###########################################################################

prefix=/usr
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: libssh2
URL: https://www.libssh2.org/
Description: Library for SSH-based communication
Version: 1.11.0
Requires.private: libssl libcrypto zlib
Libs: -L${libdir} -lssh2  -L/usr/lib -lssl -L/usr/lib -lcrypto
Libs.private:  -L/usr/lib -lssl -L/usr/lib -lcrypto
Cflags: -I${includedir}
bagder commented 4 months ago

It does indeed seem wrong and it proves I was right (and this is not a curl regression).

vszakats commented 4 months ago

Possibly related fix in libssh2 (post 1.11.0, not in a release yet): https://github.com/libssh2/libssh2/commit/624abe275fbb88816539fa5d7ebfef7f0e5ca2f1 https://github.com/libssh2/libssh2/pull/1119

kpcyrd commented 4 months ago

@vszakats thanks! I've locally changed the libssh2 PKGBUILD to use 624abe275fbb88816539fa5d7ebfef7f0e5ca2f1 (the patch doesn't apply cleanly to the latest libssh2 release) and the pkg-config file looks better now:

/usr/lib/pkgconfig/libssh2.pc

###########################################################################
# libssh2 installation details
#
# Copyright (C) The libssh2 project and its contributors.
# SPDX-License-Identifier: BSD-3-Clause
###########################################################################

prefix=/usr
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: libssh2
URL: https://www.libssh2.org/
Description: Library for SSH-based communication
Version: 1.11.1_DEV
Requires: 
Requires.private: libssl,libcrypto,zlib
Libs: -L${libdir} -lssh2 
Libs.private:  -L/usr/lib -lssl -L/usr/lib -lcrypto -L/usr/lib -lz
Cflags: -I${includedir}

The diff between the two looks like this:

> #
> # Copyright (C) The libssh2 project and its contributors.
> # SPDX-License-Identifier: BSD-3-Clause
13,16c16,20
< Version: 1.11.0
< Requires.private: libssl libcrypto zlib
< Libs: -L${libdir} -lssh2  -L/usr/lib -lssl -L/usr/lib -lcrypto
< Libs.private:  -L/usr/lib -lssl -L/usr/lib -lcrypto
---
> Version: 1.11.1_DEV
> Requires: 
> Requires.private: libssl,libcrypto,zlib
> Libs: -L${libdir} -lssh2 
> Libs.private:  -L/usr/lib -lssl -L/usr/lib -lcrypto -L/usr/lib -lz

(It seems weird there's no spaces in Requires.private, so , instead of ,, I didn't see this with other software before)

I couldn't test this with curl however since they aren't compatible (it seems libssh2_session_callback_set2 was removed):

  CC       vquic/libcurl_la-vquic.lo
  CC       vquic/libcurl_la-vquic-tls.lo
  CC       vssh/libcurl_la-libssh.lo
  CC       vssh/libcurl_la-libssh2.lo
/build/curl-rustls/src/curl/lib/vssh/libssh2.c: In function 'ssh_connect':
/build/curl-rustls/src/curl/lib/vssh/libssh2.c:3313:5: error: implicit declaration of function 'libssh2_session_callback_set2'; did you mean 'libssh2_session_callback_set'? [-Wimplicit-function-declaration]
 3313 |     libssh2_session_callback_set2(sshc->ssh_session,
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |     libssh2_session_callback_set
/build/curl-rustls/src/curl/lib/vssh/libssh2.c:3315:36: error: 'libssh2_cb_generic' undeclared (first use in this function)
 3315 |                                   (libssh2_cb_generic *)ssh_tls_recv);
      |                                    ^~~~~~~~~~~~~~~~~~
/build/curl-rustls/src/curl/lib/vssh/libssh2.c:3315:36: note: each undeclared identifier is reported only once for each function it appears in
/build/curl-rustls/src/curl/lib/vssh/libssh2.c:3315:56: error: expected expression before ')' token
 3315 |                                   (libssh2_cb_generic *)ssh_tls_recv);
      |                                                        ^
/build/curl-rustls/src/curl/lib/vssh/libssh2.c:3318:56: error: expected expression before ')' token
 3318 |                                   (libssh2_cb_generic *)ssh_tls_send);
      |                                                        ^
make[2]: *** [Makefile:3378: vssh/libcurl_la-libssh2.lo] Error 1
make[2]: Leaving directory '/build/curl-rustls/src/build-curl/lib'
make[1]: *** [Makefile:1540: all] Error 2
make[1]: Leaving directory '/build/curl-rustls/src/build-curl/lib'
make: *** [Makefile:1224: all-recursive] Error 1
vszakats commented 4 months ago

libssh2_session_callback_set2 is new to libssh2-1.11.1-DEV. curl uses it when this version is detected, but it needs the latest libssh2-DEV libraries and all headers too. It works for me and in CI. Have you tried with a vanilla libssh2-DEV build?

As for the comma, according to docs, it seems to be the "official" separator (also used by libssh2 CMake builds): https://github.com/libssh2/libssh2/commit/7f83de14a022f60c287361293f9ae513c9443d9c If here is further info or tricks to know about, let me know.