ExtUtils::Liblist->ext() finds wrong libraries that do not work for build-time linking ('-ltinfo' found at .../libtinfo.so.6.4), breaks builds (Term::ReadLine::Gnu)

[drhpc]

Module: ExtUtils::LibList

Description

I had a build failure of Term::ReadLine::Gnu in an environment consisting of a pkgsrc prefix on top of a base Debian 12 install that has some development libraries for bootstrapping, but not all, namely not libncurses-dev installed. This means there is a libtinfo.so.6.4 but no libtinfo.so in the base system.

Pkgsrc installs libncurses.so, but no libtinfo or libtermcap. It builds its package devel/p5-Term-ReadLine-Gnu to link with the full ncurses installation in the pkgsrc prefix, not the run-time-only library from Debian.

This is down to this block in the library search helper employed by the Makefile.PL:

https://github.com/Perl/perl5/blob/4e75bce0a8c85f0b9b5f94f5f6607ae84de674dd/cpan/ExtUtils-MakeMaker/lib/ExtUtils/Liblist/Kid.pm#L132

It locates libraries with names like libtinfo.so.1.2.3 and assumes that those will be found by the linker when given -ltinfo. I am not sure for which system/toolchain this assumption holds true. It does not for the GNU/Linux installs I know.

I got such a printout after adding some debugging statements myself and calling ext with debugging output enabled:

searching for all term libs
search_lib(-ltermcap)
Potential libraries are '-ltermcap':
termcap not found in /tmp/bw/devel/p5-Term-ReadLine-Gnu/work/.buildlink/lib
termcap not found in /sw/env/gcc-13.2.0/openmpi/4.1.6/lib
termcap not found in /sw/compiler/gcc-13.2.0/lib
termcap not found in /usr/local/lib
termcap not found in /usr/lib/x86_64-linux-gnu
termcap not found in /usr/lib
termcap not found in /lib/x86_64-linux-gnu
termcap not found in /lib/../lib
termcap not found in /usr/lib/x86_64-linux-gnu
termcap not found in /usr/lib/../lib
termcap not found in /lib
termcap not found in /usr/lib
termcap not found in /usr/local/lib
termcap not found in /lib64
termcap not found in /usr/lib64
Warning (mostly harmless): No library found for -ltermcap
looking for -ltermcap as (libtermcap.a / libtermcap.so) in: /sw/env/gcc-13.2.0_openmpi-4.1.6/pkgsrc/2023Q4/lib
-ltermcap not found
search_lib(-ltinfo)
Potential libraries are '-ltinfo':
tinfo not found in /tmp/bw/devel/p5-Term-ReadLine-Gnu/work/.buildlink/lib
tinfo not found in /sw/env/gcc-13.2.0/openmpi/4.1.6/lib
tinfo not found in /sw/compiler/gcc-13.2.0/lib
tinfo not found in /usr/local/lib
'-ltinfo' found at /usr/lib/x86_64-linux-gnu/libtinfo.so.6.4
/sw/compiler/gcc-13.2.0/lib/gcc/x86_64-pc-linux-gnu/13.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: /tmp/ccfeIXuF.o: undefined reference to symbol 'tgetstr'
/sw/compiler/gcc-13.2.0/lib/gcc/x86_64-pc-linux-gnu/13.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: /sw/env/gcc-13.2.0_openmpi-4.1.6/pkgsrc/2023Q4/lib/libncurses.so.6: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status

The Makefile.PL calls search_lib() in a loop over various candidates for terminfo (tinfo, termcap, ncurses, curses). The bogus early return with the bad tinfo library prevents the proper library from being found, and this somehow results in he build continuing with now -ltinfo or equivalent at all, which may be an additional bug, but not here in the core Perl modules.

Then the build errors out early because it failed to determine the readline version. The reason for that exact failure is the direct usage of tgetstr() in test code that here is only linked with libreadline and not with libncurses, which provides the symbol. This error is hidden by older binutils which are happy with the indirect loading of libncurses via libreadline. In modern times, this behaviour is punished.

When the Makefile.PL finds libncurses, it happily uses that together with libreadline and the version test as well as the build succeed.

Steps to Reproduce

Get a binary GNU/Linux distro with perl and libncurses, but not the ncurses development package installed, get the source distribution of Term::ReadLine::Gnu, run Makefile.PL.

My actual steps involve said base Linux distro and a bootstrapped pkgsrc on top. In fact there also is a self-built compiler and binutils in the game. But as the expected linker behaviour in the absence of a libtinfo.so seems obvious to me, this generic outline is hopefully enough.

It is a complicated path on which I arrived at that simple problem;-)

Expected behavior

A build ending with this linker bit:

gcc ...  -o blib/arch/auto/Term/ReadLine/Gnu/Gnu.so    -L/my/prefix/lib -lreadline -lncurses

(and succeeding)

Perl configuration

Summary of my perl5 (revision 5 version 38 subversion 2) configuration:

  Platform:
    osname=linux
    osvers=5.15.162-rrz1
    archname=x86_64-linux-thread-multi

I spare the full perl configuration output with lots of path settings specific to my system. I looked up the current relevant code in he Perl 5 git repo and have seen that it is unchanged from the version I have. This is a general logic issue in the library search and no specific bug in my install. Well, I at least think it is …

Is there a system nowadays where invoking the compiler/linker with -lfoo will locate and use libfoo.so.1.2.3 directly, without libfoo.so (or libfoo.a) being present in some form? I imagine this causing more issues now with indirect linkage not being tolerated anymore by newer binutils installs.

[iabyn]

On Thu, Jul 11, 2024 at 02:20:40PM -0700, Dr. Thomas Orgis wrote:

Module: ExtUtils::LibList

This module is part of the ExtUtils-MakeMaker distribution, which is maintained separately from perl.

You may wish to report the issue at: https://rt.cpan.org/Public/Dist/Display.html?Name=ExtUtils-MakeMaker

-- "Do not dabble in paradox, Edward, it puts you in danger of fortuitous wit." -- Lady Croom, "Arcadia"

[drhpc]

OK, I was unsure about core modules. I can send this to MakeMaker's rt, then.

[drhpc]

https://rt.cpan.org/Ticket/Display.html?id=154351