curl / curl

A command line tool and library for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET, TFTP, WS and WSS. libcurl offers a myriad of powerful features
https://curl.se/
Other
35.94k stars 6.43k forks source link

[Compile-related improvement/suggestion] Please provide a bit more information on the commandline when there are issues with librares at link-time not available at run-time #6055

Closed rubyFeedback closed 4 years ago

rubyFeedback commented 4 years ago

Hello curl developers and maintainers,

I will soon describe a bit of the issue at hand, but let me just copy/paste what I did on my linux system (tarball I use for curl at the time of writing this issue is:

http://curl.haxx.se/download/curl-7.72.0.tar.bz2

)

Next using this configure-command: ./configure --prefix=/usr

checking whether to enable maintainer-specific portions of Makefiles... no
checking whether make supports nested variables... yes
checking whether to enable debug build options... no
checking whether to enable compiler optimizer... (assumed) yes
checking whether to enable strict compiler warnings... no
checking whether to enable compiler warnings as errors... no
checking whether to enable curl debug memory tracking... no
checking whether to enable hiding of library internal symbols... yes
checking whether to enable c-ares for DNS lookups... no
checking whether to disable dependency on -lrt... (assumed no)
checking whether to enable ESNI support... no
checking for path separator... :
checking for sed... /usr/bin//sed
checking for grep... /usr/bin//grep
checking for egrep... /usr/bin//grep -E
checking for ar... /usr/bin//ar
checking for a BSD-compatible install... /usr/bin//ginstall -c
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking how to run the C preprocessor... gcc -E
checking for a sed that does not truncate output... (cached) /usr/bin//sed
checking for code coverage support... no
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin//mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports the include directive... yes (GNU style)
checking dependency style of gcc... gcc3
checking curl version... 7.72.0
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking for grep that handles long lines and -e... (cached) /usr/bin//grep
checking for egrep... /usr/bin//grep -E
checking if OS is AIX (to define _ALL_SOURCE)... no
checking if _THREAD_SAFE is already defined... no
checking if _THREAD_SAFE is actually needed... no
checking if _THREAD_SAFE is onwards defined... no
checking if _REENTRANT is already defined... no
checking if _REENTRANT is actually needed... no
checking if _REENTRANT is onwards defined... no
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking how to print strings... printf
checking for a sed that does not truncate output... (cached) /usr/bin//sed
checking for fgrep... /usr/bin//grep -F
checking for ld used by gcc... /usr/bin//ld
checking if the linker (/usr/bin//ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin//nm -B
checking the name lister (/usr/bin//nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin//ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... dlltool
checking how to associate runtime and link libraries... printf %s\n
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin//nm -B output from gcc object... ok
checking for sysroot... no
checking for a working dd... /usr/bin//dd
checking how to truncate binary pipes... /usr/bin//dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin//ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking whether to build shared libraries with -version-info... yes
checking whether to build shared libraries with -no-undefined... no
checking whether to build shared libraries with -mimpure-text... no
checking whether to build shared libraries with PIC... yes
checking whether to build static libraries with PIC... yes
checking whether to build shared libraries only... no
checking whether to build static libraries only... no
checking for inline... inline
checking if cpp -P is needed... yes
checking if cpp -P works... yes
checking if compiler is DEC/Compaq/HP C... no
checking if compiler is HP-UX C... no
checking if compiler is IBM C... no
checking if compiler is Intel C... no
checking if compiler is clang... no
checking if compiler is GNU C... yes
checking if compiler is LCC... no
checking if compiler is SGI MIPSpro C... no
checking if compiler is SGI MIPS C... no
checking if compiler is SunPro C... no
checking if compiler is Tiny C... no
checking if compiler is Watcom C... no
checking if compiler accepts some basic options... yes
configure: compiler options added: -Werror-implicit-function-declaration 
checking if compiler accepts debug disabling options... yes
configure: compiler options added: 
checking if compiler optimizer assumed setting might be used... no
checking if compiler accepts strict warning options... yes
configure: compiler options added: -Wno-system-headers 
checking if compiler halts on compilation errors... yes
checking if compiler halts on negative sized arrays... yes
checking if compiler halts on function prototype mismatch... yes
checking if compiler supports hiding library internal symbols... yes
checking for windows.h... no
checking whether build target is a native Windows one... no
checking whether build target supports WIN32 file API... no
checking for good-to-use Mac CFLAGS... no
checking to see if the compiler supports __builtin_available()... no
checking whether to support http... yes
checking whether to support ftp... yes
checking whether to support file... yes
checking whether to support ldap... yes
checking whether to support ldaps... yes
checking whether to support rtsp... yes
checking whether to support proxies... yes
checking whether to support dict... yes
checking whether to support telnet... yes
checking whether to support tftp... yes
checking whether to support pop3... yes
checking whether to support imap... yes
checking whether to support smb... yes
checking whether to support smtp... yes
checking whether to support gopher... yes
checking whether to support mqtt... no
checking whether to provide built-in manual... yes
checking whether to enable generation of C code... yes
checking whether to use libgcc... no
checking if X/Open network library is required... no
checking for gethostbyname... yes
checking for windows.h... (cached) no
checking for winsock.h... (cached) no
checking for winsock2.h... (cached) no
checking for proto/bsdsocket.h... no
checking for connect in libraries... yes
checking whether time.h and sys/time.h may both be included... yes
checking for sys/types.h... (cached) yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking time.h usability... yes
checking time.h presence... yes
checking for time.h... yes
checking for monotonic clock_gettime... yes
checking for clock_gettime in libraries... no additional lib required
checking if monotonic clock_gettime works... yes
checking for pkg-config... /usr/bin//pkg-config
checking for zlib options with pkg-config... found
checking zlib.h usability... yes
checking zlib.h presence... yes
checking for zlib.h... yes
configure: found both libz and libz.h header
checking for BrotliDecoderDecompress in -lbrotlidec... yes
checking brotli/decode.h usability... yes
checking brotli/decode.h presence... yes
checking for brotli/decode.h... yes
checking for ZSTD_createDStream in -lzstd... yes
checking zstd.h usability... yes
checking zstd.h presence... yes
checking for zstd.h... yes
checking for lber.h... yes
checking for ldap.h... yes
checking for ldapssl.h... no
checking for ldap_ssl.h... no
checking for LDAP libraries... -lldap -llber
checking for ldap_url_parse... yes
checking for ldap_init_fd... yes
checking whether to enable IPv6... yes
checking if struct sockaddr_in6 has sin6_scope_id member... yes
checking if argv can be written to... yes
checking if GSS-API support is requested... no
checking whether to enable Windows native SSL/TLS (Windows native builds only)... no
checking whether to enable Secure Transport... no
checking whether to enable Amiga native SSL/TLS (AmiSSL)... no
checking for pkg-config... (cached) /usr/bin//pkg-config
checking for openssl options with pkg-config... found
configure: pkg-config: SSL_LIBS: "-lssl -lcrypto"
configure: pkg-config: SSL_LDFLAGS: "-L/usr//lib64"
configure: pkg-config: SSL_CPPFLAGS: "-I/usr//include"
checking for HMAC_Update in -lcrypto... yes
checking for SSL_connect in -lssl... yes
checking openssl/x509.h usability... yes
checking openssl/x509.h presence... yes
checking for openssl/x509.h... yes
checking openssl/rsa.h usability... yes
checking openssl/rsa.h presence... yes
checking for openssl/rsa.h... yes
checking openssl/crypto.h usability... yes
checking openssl/crypto.h presence... yes
checking for openssl/crypto.h... yes
checking openssl/pem.h usability... yes
checking openssl/pem.h presence... yes
checking for openssl/pem.h... yes
checking openssl/ssl.h usability... yes
checking openssl/ssl.h presence... yes
checking for openssl/ssl.h... yes
checking openssl/err.h usability... yes
checking openssl/err.h presence... yes
checking for openssl/err.h... yes
checking for RAND_egd... no
checking for SSLv2_client_method... no
checking for OpenSSL_version... yes
checking for BoringSSL... no
checking for libressl... no
checking for OpenSSL >= v3... no
configure: Added /usr//lib64 to CURL_LIBRARY_PATH
checking for OpenSSL headers version... 1.1.1 - 0x1010108fL
checking for OpenSSL library version... 1.1.1
checking for OpenSSL headers and library versions matching... yes
checking for "/dev/urandom"... yes
checking for SRP_Calc_client_key in -lcrypto... yes
configure: built with one SSL backend
checking default CA cert bundle/path... /etc/ssl/certs/ca-certificates.crt
checking whether to use builtin CA store of SSL library... no
checking for library containing psl_builtin... -lpsl
checking for RTMP_Init in -lrtmp... no
checking whether versioned symbols are wanted... no
checking whether to enable Windows native IDN (Windows native builds only)... no
checking whether to build with libidn2... (assumed) yes
checking for pkg-config... (cached) /usr/bin//pkg-config
checking for libidn2 options with pkg-config... found
configure: pkg-config: IDN_LIBS: "-lidn2"
configure: pkg-config: IDN_LDFLAGS: ""
configure: pkg-config: IDN_CPPFLAGS: ""
configure: pkg-config: IDN_DIR: ""
checking if idn2_lookup_ul can be linked... yes
checking idn2.h usability... yes
checking idn2.h presence... yes
checking for idn2.h... yes
checking for pkg-config... (cached) /usr/bin//pkg-config
checking for libnghttp2 options with pkg-config... found
configure: -l is -lnghttp2
configure: -I is 
configure: -L is 
checking for nghttp2_session_set_local_window_size in -lnghttp2... yes
checking nghttp2/nghttp2.h usability... yes
checking nghttp2/nghttp2.h presence... yes
checking for nghttp2/nghttp2.h... yes
checking for pkg-config... (cached) /usr/bin//pkg-config
checking for libngtcp2 options with pkg-config... no
checking for pkg-config... (cached) /usr/bin//pkg-config
checking for quiche options with pkg-config... no
checking for pkg-config... (cached) /usr/bin//pkg-config
checking for fish options with pkg-config... no
checking for ANSI C header files... (cached) yes
checking for malloc.h... yes
checking for memory.h... no
checking for sys/types.h... (cached) yes
checking for sys/time.h... (cached) yes
checking for sys/select.h... yes
checking for sys/socket.h... yes
checking for sys/ioctl.h... yes
checking for sys/uio.h... yes
checking for assert.h... yes
checking for unistd.h... (cached) yes
checking for stdlib.h... (cached) yes
checking for arpa/inet.h... yes
checking for net/if.h... yes
checking for netinet/in.h... yes
checking for netinet/in6.h... no
checking for sys/un.h... yes
checking for linux/tcp.h... yes
checking for netinet/tcp.h... yes
checking for netdb.h... yes
checking for sys/sockio.h... no
checking for sys/stat.h... (cached) yes
checking for sys/param.h... yes
checking for termios.h... yes
checking for termio.h... yes
checking for sgtty.h... yes
checking for fcntl.h... yes
checking for alloca.h... yes
checking for time.h... (cached) yes
checking for io.h... no
checking for pwd.h... yes
checking for utime.h... yes
checking for sys/utime.h... no
checking for sys/poll.h... yes
checking for poll.h... yes
checking for socket.h... no
checking for sys/resource.h... yes
checking for libgen.h... yes
checking for locale.h... yes
checking for errno.h... yes
checking for stdbool.h... yes
checking for arpa/tftp.h... yes
checking for sys/filio.h... no
checking for sys/wait.h... yes
checking for setjmp.h... yes
checking for an ANSI C-conforming const... yes
checking for compiler support of C99 variadic macro style... yes
checking for compiler support of old gcc variadic macro style... yes
checking for size_t... yes
checking whether time.h and sys/time.h may both be included... (cached) yes
checking for sys/types.h... (cached) yes
checking for sys/time.h... (cached) yes
checking for time.h... (cached) yes
checking for sys/socket.h... (cached) yes
checking for struct timeval... yes
checking run-time libs availability... failed
configure: error: one or more libs available at link-time are not available run-time. Libs used at link-time: -lnghttp2 -lidn2 -lpsl -lssl -lcrypto -lssl -lcrypto -lldap -llber -lzstd  -lbrotlidec  -lz 
./configure

So, cutting this off here, that was the error:

configure: error: one or more libs available at link-time are not available run-time. Libs used at link-time: -lnghttp2 -lidn2 -lpsl -lssl -lcrypto -lssl -lcrypto -lldap -llber -lzstd -lbrotlidec -lz

I am compiling everything from source on linux here, using mostly the LFS/BLFs approach. So my system is quite a mess at times. :) In particular older libraries may still exist at /usr/lib/ usually.

The above "one or more libs are not available" message is quite useless, though. Because I have to ask ... WHICH libraries exactly? What is the offender? To me it is not clear, and the configure error is a bit unusual too. Isn't it normally the case that configure tells you what the problem is? Such as "nghttp2 is missing" or something like that? Instead I get a dump shown like this:

-lnghttp2 -lidn2 -lpsl -lssl -lcrypto -lssl -lcrypto -lldap -llber -lzstd -lbrotlidec -lz

So, 12 different libaries I guess, in perhaps 9 different programs or so. All of these I already have registered locally and can compile or re-compile, but this configure line is unclear to me right now. And I think it is not a good idea to just dump it on the commandline and let the user find out, or go through config.log. The latter is more than 4000 lines long and only a few lines contain what is helpful to the issue at hand.

Partially this is due to GNU autoconfigure I guess. I am not proposing to move towards cmake or meson/ninja per se either, even though I think both are better, but transition periods can often lead to other problems, so that is not a trivial task. But the current error dump is quite useless too. Nobody wants to go through the macro-stuff in configure (the KDE project moved to cmake largely to avoid this macro-stuff, understandably so), but if perhaps in the next months or years someone finds time to reconsider the error messages, it would be greatly appreciated. Specifically I think a simple change may be to just show more information here. Like, rather than list all these 12 libraries as a dump, to specifically say which one is the troublemaker - then I can concentrate on that part. Right now I am guessing ... and I already guessed wrong, since the error message was repeated. And it's still not any more helpful. :\

I was able to compile curl just fine before, so I think this is something wrong in the current set of programs I use - but without being told where the error is, I am operating mostly in a black box, which is a bit annoying.

Edit: Actually, config.log showed at the least one error:

/usr/lib/libbrotlidec.so: undefined reference to `_kBrotliPrefixCodeRanges'

However had, I still decided to file this issue request, because I think it would be better to not show a dump per se (only), and instead also report something like either that error message, or alternatively an additional line. And perhaps other users find it, and they may also have encountered the same problem, so it may be helpful to these - but please feel free to close the issue here at any moment in time; many projects like to keep the bugtracker clean, which I can understand.

jzakrzewski commented 4 years ago

As far as I can tell this error comes purely from configure. I had it in other projects also and always ask myself "why the heck does it care about runtime". I have no idea if that could be remedied.

I am not proposing to move towards cmake or meson/ninja per se either, even though I think both are better,

Just to let you know - CMake build actually is there. I don't know if I would build for LFS with it but I do build local versions for various projects.

bagder commented 4 years ago

We provide all the info we have. We don't know which specific library that's missing, only that we can't run a dead simple program that uses the libraries you've provided so far. Finding the offending library in config.log has rarely been a problem the times I've done it.

If anyone can come up with a better way or more helpful error message, I'm all ears.

configure cares about the run-time libraries because it does a few tests by building a test application and running it, and running it means it'll link with the given libraries.

bagder commented 4 years ago

This is not really a bug and we're always interested in enhancements anyway, closing this entry.