Open cstrahan opened 4 years ago
Marking as a bug, but this can't be fixed in rules_go without a corresponding fix in the Go toolchain. Consider filing a bug in golang/go for that.
cgo would need to be modified to accept response files. The Go compiler and linker already accept response files, but they're the only tools that do.
The Go linker might also need to be modified to use response files when invoking the C linker. I'm not sure if it does that already.
I can't promise this work will get priority. 128 KB is a very large limit. Can you do anything to shorten these flags? I see a lot of -Wl,--no-whole-archive -Wl,--whole-archive
pairs; do those do anything without a library in between? Can these C++ libraries be linked into a single file that can be passed to the linker later?
cgo would need to be modified to accept response files. The Go compiler and linker already accept response files, but they're the only tools that do.
We'll double check, but @dontlaugh has a patch he's working on that puts all of the ldflags in a response file, and he reports that cgo
doesn't complain and the build appears to progress further. If I'm wrong about that, and cgo
indeed does not support response files, I'll open an issue against go.
I see a lot of -Wl,--no-whole-archive -Wl,--whole-archive pairs; do those do anything without a library in between?
No, but that's coming from the approach in my other PR: https://github.com/bazelbuild/rules_go/pull/2584
If we wrap two libs A and B with --whole-archive
/--no-whole-archive
, we get:
--whole-archive A --no-whole-archive -Wl,--whole-archive B --no-whole-archive
In that PR, the logic looks something like so:
def _library_args(lib_file, alwayslink):
if not alwayslink:
return [lib_file.path]
if use_gnu_style_flags:
return ["-Wl,--whole-archive", lib_file.path, "-Wl,--no-whole-archive"]
else:
return ["-Wl,-force_load", lib_file.path]
Having _library_args
inspect a single lib makes for fairly straightforward code. Alternatively, I could change the logic such that we keep track of transitions (in the loop over each lib) from libs that have alwayslink
to those that don't, so that as few --whole-archive
/--no-whole-archive
flags are emitted as possible, though I suspect that'll make the code at least a little less obvious / more complex. I think I'll give that a go and update the aforementioned PR. Of course, I'm only really interested in implementing that logic under the assumption that we can ultimately get that upstreamed here.
Can these C++ libraries be linked into a single file that can be passed to the linker later?
I'm not sure. Bazel's C / C++ Rules docs don't suggest there's a builtin way to link a given library (or libraries) and the transitive dependencies, but I suppose I could try writing my own rule, drawing inspiration from the examples. If you happen to know a builtin way to do so, please let me know.
I'm not sure. Bazel's C / C++ Rules docs don't suggest there's a builtin way to link a given library (or libraries) and the transitive dependencies, but I suppose I could try writing my own rule, drawing inspiration from the examples. If you happen to know a builtin way to do so, please let me know.
cc_binary
has linkstatic
and linkshared
attributes that might do the trick. The resulting file might need to be passed through cc_import
before it can be used in cdeps
. Alternatively, go_binary
could be build with linkmode = "c-archive"
or linkmode = "c-shared"
, then that could be passed to a top-level cc_binary
.
Hi @cstrahan: Can you share your workaround for your 'gm-proxy' with the ego extension?
What version of rules_go are you using?
0.19.10
What version of gazelle are you using?
0.19.0
What version of Bazel are you using?
3.1.0
Does this issue reproduce with the latest releases of all the above?
Not sure. Can't build our project with the latest dependencies.
What operating system and processor architecture are you using?
x86_64 Linux
Intel(R) Core(TM) i7-6850K CPU @ 3.60GHz
Any other potentially useful information about your toolchain?
N/A
What did you do?
Attempted a build with a large C++ cgo dependency (the Envoy proxy).
What did you expect to see?
Expected the build to succeed.
What did you see instead?
The following error:
The build fails due to the
CGO_LDFLAGS
environment variable exceeding the hard 128kB limit on Linux. The build fails when invokingcgo
here:https://github.com/bazelbuild/rules_go/blob/v0.19.10/go/tools/builders/cgo2.go#L136
Here's what the environ look like up to that point (formatted as a Go literal for convenience of testing):