Different libc and ld?

[amlamarra]

Hey, I'd love to use this, but it's not working. I'm trying to emulate the Linksys E1200 firmware (and some others). After building and copying the necessary files to the root of the filesystem, I get errors indicating that it's looking for a different version of libc and ld than what's on the filesystem. Here's a look at what I'm doing:

andrew ~/nvram-faker (master) $ ./buildmipsel.sh 
/usr/bin/mipsel-linux-gnu-gcc -Wall -I./contrib/inih -ggdb -DINI_MAX_LINE=2000 -DINI_USE_STACK=0 -fPIC -c -o nvram-faker.o nvram-faker.c
make -C ./contrib/inih ini.o
make[1]: Entering directory '/home/andrew/nvram-faker/contrib/inih'
/usr/bin/mipsel-linux-gnu-gcc -ggdb -DINI_MAX_LINE=2000 -DINI_USE_STACK=0 -fPIC -c -o ini.o ini.c
make[1]: Leaving directory '/home/andrew/nvram-faker/contrib/inih'
cp ./contrib/inih/ini.o .
/usr/bin/mipsel-linux-gnu-gcc -shared -o libnvram-faker.so nvram-faker.o ini.o -Wl,-nostdlib

andrew ~/E1200 $ ls
FW_E1200_2.0.11.001_US_20180910.bin  _FW_E1200_2.0.11.001_US_20180910.bin.extracted

andrew ~/E1200 $ mv _FW_E1200_2.0.11.001_US_20180910.bin.extracted/squashfs-root/ .

andrew ~/E1200 $ cd squashfs-root/

andrew ~/E1200/squashfs-root $ cp ~/nvram-faker/libnvram-faker.so ~/nvram-faker/nvram.ini .

andrew ~/E1200/squashfs-root $ sudo chroot . ./qemu-mipsel-static -E LD_PRELOAD="./libnvram-faker.so" usr/sbin/httpd -h
usr/sbin/httpd: can't load library 'libc.so.6'

andrew ~/E1200/squashfs-root $ cp /usr/mipsel-linux-gnu/lib/libc-2.28.so lib/libc.so.6

andrew ~/E1200/squashfs-root $ sudo chroot . ./qemu-mipsel-static -E LD_PRELOAD="./libnvram-faker.so" usr/sbin/httpd -h
usr/sbin/httpd: can't load library 'ld.so.1'

andrew ~/E1200/squashfs-root $ cp /usr/mipsel-linux-gnu/lib/ld-2.28.so lib/ld.so.1

andrew ~/E1200/squashfs-root $ sudo chroot . ./qemu-mipsel-static -E LD_PRELOAD="./libnvram-faker.so" usr/sbin/httpd -h

usr/sbin/httpd: can't handle reloc type 0x2f

Any idea on how to resolve this?

BTW, I had to update the buildmipsel.sh to replace every instance of mipsel-linux- wth mipsel-linux-gnu-:

#!/bin/sh

export ARCH=mipsel
TARGET=$1

# Sets up toolchain environment variables for various mips toolchain

warn()
{
    echo "$1" >&2
}

if [ ! -z $(which mipsel-linux-gnu-gcc) ];
then
    export CC=$(which mipsel-linux-gnu-gcc)
else
    warn "Not setting CC: can't locate mipsel-linux-gnu-gcc."
fi

if [ ! -z $(which mipsel-linux-gnu-ld) ];
then
    export LD=$(which mipsel-linux-gnu-ld)
else
    warn "Not setting LD: can't locate mipsel-linux-gnu-ld."
fi

if [ ! -z $(which mipsel-linux-gnu-ar) ];
then
    export AR=$(which mipsel-linux-gnu-ar)
else
    warn "Not setting AR: can't locate mipsel-linux-gnu-ar."
fi

if [ ! -z $(which mipsel-linux-gnu-strip) ];
then
    export STRIP=$(which mipsel-linux-gnu-strip)
else
    warn "Not setting STRIP: can't locate mipsel-linux-gnu-strip."
fi

if [ ! -z $(which mipsel-linux-gnu-nm) ];
then
    export NM=$(which mipsel-linux-gnu-nm)
else
    warn "Not setting NM: can't lcoate mipsel-linux-gnu-nm."
fi

make $TARGET || exit $?

[zcutlip]

Looks like the version of the toolchain you're using in incompatible with the version used to build your firmware. Are you able to build a simple helloworld and run it on that firmware?

GCC toolchain versions are notoriously picky about forward/backward compatibility. It's also pretty picky about what toolchain versions will build what other toolchain versions. I used to collect and hoard gcc toolchains for this reason, because sometimes it would be trial-and-error finding one that would build binaries that would run on a given firmware.

[amlamarra]

Sorry. I'm not much of a software developer. I just installed this package on my system: https://packages.debian.org/buster/gcc-mipsel-linux-gnu

That's what allowed me to cross compile to mips. A simple helloworld does work:

andrew ~/E1200/squashfs-root $ cat hello.c 
#include <stdio.h>
int main() {
    printf("Hello world\n");
    return 0;
}

andrew ~/E1200/squashfs-root $ mipsel-linux-gnu-gcc hello.c -o hello

andrew ~/E1200/squashfs-root $ file hello
hello: ELF 32-bit LSB shared object, MIPS, MIPS32 rel2 version 1 (SYSV), dynamically linked, interpreter /lib/ld.so.1, BuildID[sha1]=dd4c3c686dfbcd3513219a2cf4dd02494047321c, for GNU/Linux 3.2.0, not stripped

andrew ~/E1200/squashfs-root $ sudo chroot . ./qemu-mipsel-static ./hello
Hello world

andrew ~/E1200/squashfs-root $ sudo chroot . ./qemu-mipsel-static -E LD_PRELOAD="./libnvram-faker.so" ./hello
Hello world

[amlamarra]

Is there any way for me figure out what toolchain version was used to build the firmware? Would I need the source (if that would even tell me)?

[amlamarra]

Never mind. I found the correct toolchain and it all works now, thanks!

[YanhuiJessica]

Never mind. I found the correct toolchain and it all works now, thanks!

What is the correct toolchain?

[amlamarra]

In this particular case, it was uclibc-4.2.3.

[YanhuiJessica]

In this particular case, it was uclibc-4.2.3.

Thanks～！ʕ •ᴥ•ʔ

[YanhuiJessica]

Sorry to bother you again. After solving the above problems，the new one arose：

can't resolve symbol '__ctype_b_loc'

Did I miss sth?(╥ω╥) I used the toolchain from: https://toolchains.bootlin.com/

[amlamarra]

You've just set the LD_PRELOAD environment variable for every program you run in that terminal. When you executed ls, it first looked to /lib/libnvram-faker.so for the __ctype_b_loc symbol. Which, apparently it had but couldn't resolve properly... I'm not sure.

Anyway, you need to use libnvram-faker.so when executing something specific. In my case, I was trying to run the httpd binary from a router's firmware in a QEMU user environment like so: sudo chroot . ./qemu-mipsel-static -E LD_PRELOAD="./libnvram-faker.so" usr/sbin/httpd

[YanhuiJessica]

$ sudo chroot . ./qemu-mips-static -E LD_PRELOAD="./lib/libnvram-faker.so" ./sbin/lighttpd
./sbin/lighttpd: can't resolve symbol '__ctype_b_loc'

Sad. I give up.(╥ω╥) Maybe the toolchain is wrong again

---

Yeah~The toolchain is wrong again.