Allow C++ and Swift library binaries to be installed into a location other than the build output location

[adammurdoch]

Currently, on Linux and MacOs, we don't do anything special to allow library binaries to be relocated on the file system and so as a result they are not. This is something we have intentionally ignored for now so as to make progress on other things, as the binaries remain mostly usable while everything is happening on the build machine.

However, when publishing to a binary repository the binaries leave the build machine and typically end up in a location that is different to where they were built, rendering them unusable.

This issue also affects build caching to some degree. While the caching can happily deal with binaries that are not relocatable, it is much more effective when the binaries can be reused into different locations.

For this story, change the C++ and Swift plugins to drive the linker so that library binaries can be relocated on the file system. This is an unblocking story, so assume for this story that the libraries are used only by executables built by Gradle. Out of scope are libraries consumed in other ways or executables that are packaged in some other way than the install task.

[lacasseio]

Simply to consolidate the information from order sources, rpath and install_name_tool seems to be the common solution for this problem.

A workflow would be using the install_name_tool utility to set the libraries to use @rpath on the library (producer) side. install_name_tool is used to set the @rpath to use @loader_path on the executable (consumer side).

@ghale Would you mind sharing your reading list on this topic so we can capture the knowledge here as we develop a solution.

Resources

• https://blogs.oracle.com/dipol/dynamic-libraries,-rpath,-and-mac-os

[ghale]

This seems to be the most coherent explanation: https://matthew-brett.github.io/docosx/mac_runtime_link.html

Some other links: http://log.zyxar.com/blog/2012/03/10/install-name-on-os-x/ https://stackoverflow.com/questions/4513799/how-to-set-the-runtime-path-rpath-of-an-executable-with-gcc-under-mac-osx https://stackoverflow.com/questions/33991581/install-name-tool-to-update-a-executable-to-search-for-dylib-in-mac-os-x

[adammurdoch]

The basic problem we have here is that when you build the binaries for a library, you define where the binaries that use the binary should look for the binary, which means that at build time you need to know something of the application or applications that will be using the library. From a certain perspective this is no different to anything else we need to define at build time, such as debuggability or profiling or support for a certain ABI or whatever, and it's just another dimension to the context in which the library will be used.

There are basically 2 patterns that have emerged to deal with this:

1. A library is always installed at the same location on every machine.
2. A library is located relative to some path specified by the consuming binary.

We want to provide good support for both of these patterns, but provide option 2 as the out-of-the-box convention for the native binaries we produce. This isn't just about driving the compiler in the right way, but also affects:

• dev-time installation, as we need to know where to put the libraries on the local machine so the consuming application or test suite can find them. "Installation" here means "make usable", which includes running tests and importing into the IDE.
• dependency management, as we need to select library binaries that are compatible with how the consuming component intends to be used. For example, if I'm building an application whose install image can be relocated on the file system, I want by library binaries to also be relocatable. A library may offer relocatable and non-relocatable variants of its binaries. Or may not.

I think we should put together a design doc to describe how 'packaging' should look and and work back from there.

[swpalmer]

You are missing the 3rd pattern: The library will be at a different location on every machine, but will be reachable by the library load path for that system. E.g. %PATH% on Windows, LD_LIBRARY_PATH on Linux or macOS

I have a case where the environment is very dynamic (plugin based) and as such LD_LIBRARY_PATH is unique for different processes of the same executable that launch with different sets of plugins.

This means that libraries may not be in a fixed location, nor will they be in the same relative location.

[b-john]

Hey all-- I would also be interested in leveraging the library load path / LD_LIBRARY_PATH on Linux.

Aka, link against shared libraries with the "-l" option in gcc world instead of absolute path. Same story as others in relation to shoving all my shared libraries and executables into a "/bin" or "/lib" directory for installation with the library path set to that directory instead of mirroring the build machine layout on each installation container. Thanks! https://gcc.gnu.org/onlinedocs/gcc/Link-Options.html

[b-john]

Would an adequate design here be to modify the GccLinker to leverage "-l" instead of the full paths? https://github.com/gradle/gradle/blob/5d831756be446e8d9df7ed124230092a8e1ec0e9/subprojects/platform-native/src/main/java/org/gradle/nativeplatform/toolchain/internal/gcc/GccLinker.java#L76

Instead of doing "gcc ..... /home/me/repo/mylibrary/build/lib/main/release/libmylibrary.so" it would be "gcc .... -L /home/me/repo/mylibrary/build/lib/main/release -lmylibrary". (I believe this is a pattern leveraged by Maven NAR & bazel as well)

To my knowledge, there is not many use cases for linking against absolute paths. The above change would be non-passive in that if you wanted to build everything and run executables without setting the library path (LD_LIBRARY_PATH) it would now fail; But install takes care of that for us already by setting LD_LIBRARY_PATH.

Thanks for any thoughts!

[b-john]

Validated manually that if we made such a change ( https://github.com/b-john/gradle/commit/aa986a036cc3f52c378e5c8784aa1cd84c89f0f0 ) that it would switch from absolute paths to library path.

Appreciate any thoughts; Believe we would need to switch unit/integration tests to also set the library path to automatically launch. Will work on those changes & following contribution proceedure / change automated tests if there are no objections.

[adammurdoch]

I think it's a reasonable option to support, in general. For me, there are some questions:

Firstly, should we offer support for other patterns as well? Requiring that an environment variable be set in a particular way in order to make an application runnable can be an awkward pattern, or at least potentially unreliable, but it is very common and there are use cases for it. It would be good to offer an alternative strategy that uses relative paths and let people choose which strategy to use for their application. It would also be good to allow absolute paths as a strategy as well.

If the answer is 'yes', then another question is what should be the default?

It's probably ok to just make the change you propose for now, and figure out these questions later. Right now we support only one option and I think it would be fine to switch from using absolute paths to environment variables and then add further options later.

[b-john]

Thanks Adam! I'll plan on getting a PR out in the next few weeks.

Those are good questions; It would be interesting to see if we could identify a use case for the absolute paths. There's never been a shared library I've installed that has wanted it (LibC, OpenSSL, LDAP, Boost, anything in /lib). But it's possible which means it's likely used somewhere, and agreed could be a bit awkard if wanting to run what Gradle built manually without install.

[big-guy]

Let's split this up based on the PR

[big-guy]

Take a look at the PR

[lacasseio]

I'm looking at the PR now. I will look at writing a design document next.

[lacasseio]

I commented on the PR. On first look, it doesn't look to bring anything to what we can already do with Gradle's binaries. At the moment, the binaries are relocatable through the LD_LIBRARY_PATH environment variable.

[lacasseio]

For the sake of completeness here is the summary of the discussion we had on the topic. Binaries can 2 type of relocation: no relocation (can use absolute path) and relocatable (use relative or no path at all). There is a clear distinction to make between relocatable binaries and packaging with relocatable binaries under a specific layout. Both could use rpath or install name or binary patching or something else. The different relocation scheme can be considered within the variant set of a component. The usage of each relocation scheme can differ by build type. For example, no relocatable could be used for debugging in the IDE but not as published binaries.

One issue that may arise with https://github.com/gradle/gradle/pull/6176 is debuggability through IDE. However, this should be a separate issue where Gradle could ease the debugging process by preparing gdb or windbg script/configuration to account for the relocation scheme used.

[lacasseio]

The work specific to the PR (Linux relocation) is split out in issue https://github.com/gradle/gradle-native/issues/930.