Ncurses links with host Linker

[dslm4515]

Arch: i686 Distro: MLFS-4.00

While building the /tools toolchain, ncurses will not use the libc.so in /tools/lib

$ ldd lib/libtinfow.so.6.2
        ldd (0xb7f47000)
        libc.so => ldd (0xb7f47000)

I have tried setting these:

export LDFLAGS=-L/tools/lib
export CPPFLAGS=-I/tools/include

For configure:

./configure --prefix=/tools \
    --build=${MLFS_HOST} \
    --host=${MLFS_TARGET} \
    --with-shared   \
    --without-debug \
    --without-ada   \
    --enable-widec  \
    --enable-overwrite     \
    --with-termlib \
    --enable-termcap    \
    --libdir=/tools/lib \
    --oldincludedir=/tools/include

[firasuke]

Is this a cross compilation or a native one inside a chroot environment?

[dslm4515]

This is during cross-compilation using the mlfs user. Building the tool chain to use for the chroot environment.

I am going to try to use your mussel project to see if I get the same issue. Not sure if issue lies in a leaky cross-tool chain or the developing tool chain for chroot.

[firasuke]

I'm pretty sure it's leaking if it's relying on the host tools, double check other packages and see if they have stuff from the host as well.

[dslm4515]

I did check bash (was compiled before ncurses) and it uses the Libc in /tools/lib... although some libraries from GCC do link to libraries in /usr/lib instead of /tools/lib. So I am going rebuild both toolchains from scratch.

I had a similar issue with my raspberry pi 4b but it was easily fixed by creating /tools/etc/led-musl-i386.path

[dslm4515]

Is there a way I can test the cross-tool chain for leaks?

It's annoying when I'm building the last package for the tool chain and it fails due to a leak in either tool chain?

[firasuke]

Test it for leaks?

Use this .bashrc:

set +h
umask 022
export LC_ALL=POSIX
export PATH=/tools/bin:/usr/bin:/bin

Use this .bash_profile:

exec /usr/bin/env -i \
  HOME=/home/mlfs \
  PATH=/tools/bin:/usr/bin:/bin \
  TERM=xterm-256color \
  PS1='\u:\w\$ ' \
  /usr/bin/bash

Make sure that ARCH is correctly set as an environment variable (same goes for CROSS_COMPILE (don't forget the extra - as a suffix)).

Ensure that TOOL=/tools.

Run (where CC is the compiler you're building with, and dummy.c is any example C program):

$CC dummy.c -Wl,--verbose 2>&1 | grep succeeded

Output should be (along these lines):

checking what assembler to use... /tools/x86_64-pc-linux-musl/bin/as
checking what linker to use... /tools/x86_64-pc-linux-musl/bin/ld

Run (where READELF and CC are the ones you provided):

$READELF -l $CC | grep interpreter

Output should show the use of /tools/lib.

Run (where LD is the one you provided):

$LD --verbose | grep SEARCH

and see if the linker's search paths are correct.

Run (where CPP is the one you provided):

$CPP -x c -v

and see if it's searching for the include files in the correct dirs.

Run (where GCC is the one you provided):

$CC -print-prog-name-ld

and see if it's correct.

You can also try running ldd inside chroot and see if points to the correct paths, or LD_LIBRARY_PATH outside of chroot (or by using the correct ld-musl.path file) and see if the paths are correct.

Hope that helps.

P.S. I'm no longer using this isolated sandboxed method (cross compile a native chroot and build everything inside it) so some things could be off.

[firasuke]

I found a patch of mine that I used to apply for glaucus that might be of benefit to you (for this isolated sandboxed method that requires cross compiling a native chroot and building everything inside of it).

I've adjusted the first one for use inside mlfs, so you can try it out (for the native gcc build):

https://gist.github.com/firasuke/c60a8fe456b46371991b12288138e153

And here's its counterpart for adjusting system paths when inside chroot (I didn't bother adjusting it for mlfs as you might use it as it is; it assumes /usr/lib, so you may want to change that to /lib as I don't know what mlfs uses):

https://raw.githubusercontent.com/glaucuslinux/glaucus/forsaken/cerata/gcc/patches/glaucus/0002-adjust-system-paths-for-x86-64.patch

Hope these help as well.

Keep in mind that I no longer support this method of building stuff as it's not portable and makes the selection of architectures limited based on what you can create a native chroot environment for and run it on (you'll mostly be limited to x86-64, i686 if doable and aarch64 if doable as well).

[dslm4515]

I finished building a cross-toolchain with mussel, and I didn't realize until after the build its not using an isolated sandboxed method... so I'm not sure if i can use it.

I rebuilt my cross-toolchain and checked for leaks, using your suggestions...

Wrong "interpreter" and search directories:

$ /cross-tools/bin/i686-mlfs-linux-musl-readelf -l /cross-tools/bin/i686-mlfs-linux-musl-gcc | grep interpreter
      [Requesting program interpreter: /lib/ld-musl-i386.so.1]
$ /cross-tools/bin/i686-mlfs-linux-musl-ld --verbose | grep SEARCH | sed 's|; |\n|g'
SEARCH_DIR("=/cross-tools/i686-mlfs-linux-musl/lib32")
SEARCH_DIR("=/usr/local/lib32")
SEARCH_DIR("=/lib32")
SEARCH_DIR("=/usr/lib32")
SEARCH_DIR("=/cross-tools/i686-mlfs-linux-musl/lib")
SEARCH_DIR("=/usr/local/lib")
SEARCH_DIR("=/lib")
SEARCH_DIR("=/usr/lib");

Looks like the correct include directories:

 $ /cross-tools/bin/i686-mlfs-linux-musl-cpp -x c -v
...
#include "..." search starts here:
#include <...> search starts here:
 /mnt/mlfs/cross-tools/bin/../lib/gcc/i686-mlfs-linux-musl/10.2.0/../../../../i686-mlfs-linux-musl/include
 /mnt/mlfs/cross-tools/bin/../lib/gcc/i686-mlfs-linux-musl/10.2.0/include
 /mnt/mlfs/cross-tools/bin/../../cross-tools/usr/include
End of search list.

Linker looks correct:

$ /cross-tools/bin/i686-mlfs-linux-musl-gcc-10.2.0 -print-prog-name=ld
/mnt/mlfs/cross-tools/bin/../lib/gcc/i686-mlfs-linux-musl/10.2.0/../../../../i686-mlfs-linux-musl/bin/ld

I'll try those patches too.

But still, your suggestion are useful

[dslm4515]

Found the old tool chains from a very old successful build (4.00) and yielded same results...I suppose my cross toolchain (7.00) is on the right track. Going to build next tool chain (for chroot [sandbox] ).

[firasuke]

Glad you found my suggestions useful, let's hope you can also fix this issue and get everything working again.

[dslm4515]

Fixed issue. I rebuilt ncurses with configure options from an older build:

./configure --prefix=/tools \
            --with-shared   \
            --without-debug \
            --without-ada   \
            --enable-widec  \
            --enable-overwrite

Now the ncurses libraries don't seem to link back to host:

$ ldd /tools/lib/libncursesw.so.6.2
        ldd (0xb7ee7000)
        libc.so => ldd (0xb7ee7000)

Before, this library was linked to /usr/lib/libtinfow.so.6

Not sure why these options caused this issue: --with-termlib --enable-termcap

But now libtinfow.so.6.2 isn't built and looking at my successful build on the raspberry pi 4b, that library is not present in /tools/lib... Perhaps I forgot to change the configure options when I had this issue on the raspberry pi 4b ?

[firasuke]

Changing these options should have nothing to do with what libc ncurses links to.

I can't tell what changed based on the outputs for ldd that you provided above (they seem to be identical in the first post here and in the post before this).

[dslm4515]

@firasuke, Sorry, my first post is not as detailed as it should be. Initially, I compiled ncurses with --with-termlib --enable-termcap... then nano failed to compile due to unresolved symbols from libncursesw.so.6... so I used ldd to see what libncursesw.so.6.2 links with... it was linking to /usr/lib/libtinfow.so.6 on those host.

Just to make sure ldd functions properly, i checked bash:

$ ldd /tools/bin/bash
        /tools/lib/ld-musl-i386.so.1 (0xb7eef000)
        libc.so => /tools/lib/ld-musl-i386.so.1 (0xb7eef000)

So I recompiled ncurses again with out --with-termlib --enable-termcap and try ldd again on libncursesw.so.6.2:

$ ldd /tools/lib/libncursesw.so.6.2
        ldd (0xb7f5c000)
        libc.so => ldd (0xb7f5c000)

Output from ldd seems odd. So I try another library that works/links properly:

$ ldd /tools/lib/liblzma.so.5.2.5
        ldd (0xb7efc000)
        libc.so => ldd (0xb7efc000)

Same thing. I've been on Musl so long, I forgot how it might look with glibc.

[firasuke]

Doesn't look that suspicious to me.

[Strahinja]

I'm getting this output from ldd in this stage of the build:

$ ldd /tools/lib/libncursesw.so.6.3
    ldd (0x7fbf9bc67000)
    libc.so => ldd (0x7fbf9bc67000)
Error relocating /tools/lib/libncursesw.so.6.3: _nc_get_alias_table: symbol not found
Error relocating /tools/lib/libncursesw.so.6.3: _nc_get_hash_info: symbol not found
Error relocating /tools/lib/libncursesw.so.6.3: _nc_get_hash_table: symbol not found
Error relocating /tools/lib/libncursesw.so.6.3: _nc_get_table: symbol not found
Error relocating /tools/lib/libncursesw.so.6.3: _nc_tinfo_fkeys: symbol not found

During compilation:

( cd man && make DESTDIR="" RPATH_LIST="/tools/lib" all )
make[1]: Entering directory '/mnt/src/ncurses-6.3/man'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/mnt/src/ncurses-6.3/man'
( cd include && make DESTDIR="" RPATH_LIST="/tools/lib" all )
make[1]: Entering directory '/mnt/src/ncurses-6.3/include'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/mnt/src/ncurses-6.3/include'
( cd ncurses && make DESTDIR="" RPATH_LIST="/tools/lib" all )
make[1]: Entering directory '/mnt/src/ncurses-6.3/ncurses'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/mnt/src/ncurses-6.3/ncurses'
( cd progs && make DESTDIR="" RPATH_LIST="/tools/lib" all )
make[1]: Entering directory '/mnt/src/ncurses-6.3/progs'
x86_64-mlfs-linux-musl-gcc ../obj_s/tic.o ../obj_s/dump_entry.o ../obj_s/tparm_type.o ../obj_s/transform.o -L../lib  -DHAVE_CONFIG_H -I../progs -I. -I../include  -DNDEBUG -O2 --param max-inline-insns-single=1200  -fPIC  -L../lib  -lncursesw -lncursesw    -o tic
/mnt/cross-tools/bin/../lib/gcc/x86_64-mlfs-linux-musl/13.0.0/../../../../x86_64-mlfs-linux-musl/bin/ld: ../obj_s/tic.o: in function `check_user_capability_type':
tic.c:(.text+0x171b): undefined reference to `_nc_get_hash_table'
/mnt/cross-tools/bin/../lib/gcc/x86_64-mlfs-linux-musl/13.0.0/../../../../x86_64-mlfs-linux-musl/bin/ld: ../obj_s/tic.o: in function `check_termtype':
tic.c:(.text+0x258e): undefined reference to `_nc_tinfo_fkeys'
/mnt/cross-tools/bin/../lib/gcc/x86_64-mlfs-linux-musl/13.0.0/../../../../x86_64-mlfs-linux-musl/bin/ld: ../obj_s/dump_entry.o: in function `nametrans':
dump_entry.c:(.text+0xc67): undefined reference to `_nc_get_hash_table'
/mnt/cross-tools/bin/../lib/gcc/x86_64-mlfs-linux-musl/13.0.0/../../../../x86_64-mlfs-linux-musl/bin/ld: ../lib/libncursesw.so: undefined reference to `_nc_get_alias_table'
/mnt/cross-tools/bin/../lib/gcc/x86_64-mlfs-linux-musl/13.0.0/../../../../x86_64-mlfs-linux-musl/bin/ld: ../lib/libncursesw.so: undefined reference to `_nc_get_hash_info'
/mnt/cross-tools/bin/../lib/gcc/x86_64-mlfs-linux-musl/13.0.0/../../../../x86_64-mlfs-linux-musl/bin/ld: ../lib/libncursesw.so: undefined reference to `_nc_get_table'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:262: tic] Error 1
make[1]: Leaving directory '/mnt/src/ncurses-6.3/progs'
make: *** [Makefile:134: all] Error 2