Bug: when compiling executable, emcc "forgets" provided needed library

[Labnann]

Reproducing:

// src/foo.c
#include <stdio.h>
#include <stdlib.h>

void bar();

void foo() {
    printf("Foo \n");
    bar();
}

// src/bar.c
#include <stdio.h>
#include <stdlib.h>

void bar() {
    printf("Bar \n");
}

// src/main.c
void foo();

int main()
{
    foo();
    return 0;
}

Compilation:

emcc -o src/bar.o -c -fPIC -DPIC src/bar.c   
emcc -o bar.so -sSIDE_MODULE=2 -sEXPORTED_FUNCTIONS=_bar src/bar.o

emcc -o src/foo.o -fPIC -DPIC -c src/foo.c   
emcc -o foo.so src/foo.o -sEXPORTED_FUNCTIONS=_foo -sSIDE_MODULE=2 bar.so

emcc -o src/main.o -c -fPIC -DPIC src/main.c   
emcc -o main src/main.o -sMAIN_MODULE -sFETCH foo.so bar.so

This will throw the error:

emcc: error: foo.so: shared library dependency not found: `bar.so`

Expected: The file should compile without any errors. Since bar.so is provided.

[sbc100]

Thanks I was able to re-produce the issue.

As of today it looks like you need to add -L. so that the shared libraries resolver an find the dependency. For now I've added a test that covers this case: https://github.com/emscripten-core/emscripten/pull/21682.

Is adding -L. a suitable solution for you? That way you also just avoid specifying bar.so completely.. since the main program has not direct dependencies on it.

[sbc100]

Interesting this is also problem with native gcc/clang toolchains.

In fact in that case its not enough to add -L., you need to add -Wl,-rpath=. or -Wl,-ppath-link=..

$ gcc -shared bar.c -fPIC -o libbar.so
$ gcc -shared foo.c -fPIC -o libfoo.so libbar.so 
$ gcc main.c -L. libfoo.so 
/usr/bin/ld: warning: libbar.so, needed by libfoo.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: libfoo.so: undefined reference to `bar'
collect2: error: ld returned 1 exit status
$ gcc main.c -Wl,-rpath=. libfoo.so 

[Labnann]

Interesting this is also problem with native gcc/clang toolchains.

In fact in that case its not enough to add -L., you need to add -Wl,-rpath=. or -Wl,-ppath-link=..

$ gcc -shared bar.c -fPIC -o libbar.so
$ gcc -shared foo.c -fPIC -o libfoo.so libbar.so 
$ gcc main.c -L. libfoo.so 
/usr/bin/ld: warning: libbar.so, needed by libfoo.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: libfoo.so: undefined reference to `bar'
collect2: error: ld returned 1 exit status
$ gcc main.c -Wl,-rpath=. libfoo.so 

I tried with clang but couldn't reproduce this.

Tried both of these:

clang -shared bar.c -fPIC -o libbar.so
clang -shared foo.c -fPIC -o libfoo.so libbar.so
clang main.c libbar.so libfoo.so

And

clang -shared bar.c -fPIC -o libbar.so
clang -shared foo.c -fPIC -o libfoo.so libbar.so
clang main.c -L. libfoo.so

Both executed successfully without any issues on my end.

Here is the clang --version output:

clang version 17.0.6
Target: aarch64-unknown-linux-android24
Thread model: posix
InstalledDir: /data/data/com.termux/files/usr/bin

I shall report on gcc tomorrow, when I am using my main machine. However, as far as I remember, this error didn't occur for me before when I tried this with gcc.

[Labnann]

Thanks I was able to re-produce the issue.

As of today it looks like you need to add -L. so that the shared libraries resolver an find the dependency. For now I've added a test that covers this case: #21682.

Is adding -L. a suitable solution for you? That way you also just avoid specifying bar.so completely.. since the main program has not direct dependencies on it.

It does work. Instead of libfoo.so, I have to do -L/path/to/foo/ now. But I was wondering, if it creates bigger executable if that path contains multiple .so files?

[sbc100]

Yes, this should not effect the executable that gets built. Its just about the resolved the shared libraries names and reporting missing symbols.

[Labnann]

Interesting this is also problem with native gcc/clang toolchains. In fact in that case its not enough to add -L., you need to add -Wl,-rpath=. or -Wl,-ppath-link=..

$ gcc -shared bar.c -fPIC -o libbar.so
$ gcc -shared foo.c -fPIC -o libfoo.so libbar.so 
$ gcc main.c -L. libfoo.so 
/usr/bin/ld: warning: libbar.so, needed by libfoo.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: libfoo.so: undefined reference to `bar'
collect2: error: ld returned 1 exit status
$ gcc main.c -Wl,-rpath=. libfoo.so 

I tried with clang but couldn't reproduce this.

Tried both of these:

clang -shared bar.c -fPIC -o libbar.so
clang -shared foo.c -fPIC -o libfoo.so libbar.so
clang main.c libbar.so libfoo.so

And

clang -shared bar.c -fPIC -o libbar.so
clang -shared foo.c -fPIC -o libfoo.so libbar.so
clang main.c -L. libfoo.so

Both executed successfully without any issues on my end.

Here is the clang --version output:

clang version 17.0.6
Target: aarch64-unknown-linux-android24
Thread model: posix
InstalledDir: /data/data/com.termux/files/usr/bin

I shall report on gcc tomorrow, when I am using my main machine. However, as far as I remember, this error didn't occur for me before when I tried this with gcc.

Interesting this is also problem with native gcc/clang toolchains.

In fact in that case its not enough to add -L., you need to add -Wl,-rpath=. or -Wl,-ppath-link=..

$ gcc -shared bar.c -fPIC -o libbar.so
$ gcc -shared foo.c -fPIC -o libfoo.so libbar.so 
$ gcc main.c -L. libfoo.so 
/usr/bin/ld: warning: libbar.so, needed by libfoo.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: libfoo.so: undefined reference to `bar'
collect2: error: ld returned 1 exit status
$ gcc main.c -Wl,-rpath=. libfoo.so 

I confirm this. I was able to reproduce this in both gcc and clang in my x86_64 machine running fedora.

By the way, this error is not triggered in case of :

$CC -shared bar.c -fPIC -o libbar.so
$CC -shared foo.c -fPIC -o libfoo.so libbar.so
$CC main.c libbar.so libfoo.so

In this case, program compiles successfully without any errors. In both gcc and clang.

Here is the gcc --version output:

gcc (GCC) 13.2.1 20231011 (Red Hat 13.2.1-4)
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

I tested with 2 clang versions in my x86_64 machine and it is interesting that the problem did occur as you showed before, unlike my other machine.

Here are the two clang --version output, in which the error you mentioned occured:

clang version 16.0.5 (Fedora 16.0.5-1.fc38)
Target: x86_64-redhat-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

And

clang version 19.0.0git (https:/github.com/llvm/llvm-project 6c7805d5d186a6d1263f90b8033ad85e2d2633d7)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/labnan/start/devtools/emsdk/upstream/bin

[sbc100]

If you link with -L. does that work for you? If so I think that is best solution for now.

Also, you can probably just drop the explictly libbar.so from your command line since libfoo.so already implicitly depends on it.

[Labnann]

If you link with -L. does that work for you? If so I think that is best solution for now.

Also, you can probably just drop the explictly libbar.so from your command line since libfoo.so already implicitly depends on it.

Yes, it works. Directly using -L. does not work on all cases. Using -L/path/to/foo/ works :).

Have you considered #21605 by the way? If you can point out the issue with this patch, I will try writing a better patch that fixes this issue.