cc_library produces empty .so library if srcs input is a tree artifact (dynamic linking/gcc gold linker/Linux)

vxf commented 2 years ago

Description of the problem / feature request:

Using Linux gcc gold linker as default of running bazel in Ubuntu 20.04. If the srcs input of cc_library is a tree artifact, like a declare_directory of a rule, the generated actions for building will pass the directory of the corresponding object .o files to the linker command. This works fine for the ar command producing a correct .a, however for the .so linker command it will produce an empty .so dynamic library.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

1. Have any rule generating .c/cpp/h source files but return the object from declare_directory (for example the specific names of files are not previously known).

    out_dir = ctx.actions.declare_directory(ctx.attr.name)
(...)
    compilation_context = cc_common.create_compilation_context(
        headers = depset([out_h]),
        includes = depset([out_dir.path])
    )

    return [
        DefaultInfo(files = depset([out_dir])),
        CcInfo(compilation_context = compilation_context)
    ]

2. Have a cc_library using the output of the previous rule in the srcs to compile them

# this rules generates .cpp and .h in a tree artifact
my_best_generator(
    name = "vasco",
)

# this library uses the generated files
cc_library(
    name = "vasco_lib",
    srcs = [":vasco"],
    deps = [":vasco"],
)

3. Build the cc_library and Verify the symbols exported in the resulting .a and .so (for example nm command)

4. The .so will not have symbols from the generated sources

6. Verify the actions with aquery

For linking the .a file:

  Command Line: (exec /usr/bin/ar \
    rcsD \
    bazel-out/k8-fastbuild/bin/app/libvasco_lib.a \
    bazel-out/k8-fastbuild/bin/app/_pic_objs/vasco_lib/vasco \
    @bazel-out/k8-fastbuild/bin/app/libvasco_lib.a-2.params)

This works fine for ar.

However for linking the .so file:

  Command Line: (exec /usr/bin/gcc \
    -shared \
    -o \
    bazel-out/k8-fastbuild/bin/app/libvasco_lib.so \
    -Wl,-S \
    '-fuse-ld=gold' \
    -Wl,-no-as-needed \
    -Wl,-z,relro,-z,now \
    -B/usr/bin \
    -pass-exit-codes \
    -Wl,--start-lib \
    bazel-out/k8-fastbuild/bin/app/_pic_objs/vasco_lib/vasco \
    -Wl,--end-lib \
    -lstdc++ \
    -lm \
    @bazel-out/k8-fastbuild/bin/app/libvasco_lib.so-2.params)

This will produce an empty dynamic library, since --start-lib --end-lib doesn't expect a directory

7. tested in several versions of bazel including 4.x, 5.x and 6.x

What operating system are you running Bazel on?

Linux Ubuntu 20.04

What's the output of `bazel info release`?

release 5.0.0

Have you found anything relevant by searching the web?

Places to look:

Man page of gcc gold linker: http://manpages.ubuntu.com/manpages/trusty/man1/ld.gold.1.html
Introduction of --start-lib end --end-lib options in binutils: https://sourceware.org/pipermail/binutils/2010-March/066146.html

Any other information, logs, or outputs that you want to share?

My test scenario: test.txt

github-actions[bot] commented 1 year ago

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 14 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

cbandera commented 1 year ago

I can confirm that I run into the same issue. Is there any solution to this yet?

cbandera commented 1 year ago

I need to refine my previous post:

The aquery suggests that the whole directory is passed to the linker, but
running the buildcommand with -s reveils that in fact the object files are being passed to the linker, contrary to what the aquery said
But still, the final .so file is empty :(

cpsauer commented 1 year ago

(Probably not the answer you're hoping to hear, but my sense is that quite a few problems remain with tree artifacts--not sure how much they want to support them vs globs.)

peakschris commented 1 year ago

I am also impacted by this. I need the tree artifacts in order to compile a directory of codegen outputs, where the number/names of files is not know ahead of time. I can switch off 'supports_start_end_lib' in my custom copy of the automatic C++ toolchain as a workaround.

evan-flynn-apexai commented 5 days ago

I've also ran into this issue

bazelbuild / bazel