gradle c++ cmake-library publishing multiple libraries custom-publishing-plugin

Cmake can produce multiple library files produced by root cmakelist.txt or multiple subprojects. there is no way to set these library paths/names, but just a single library name. is it possible to do this or does this needs a feature request to the custom publish plugin. Note: using Kotlin DSL

Expected Behavior

Instead of:

cmake {
    binary.set("lib/libopencv_core.so" ))
}

Replace:

cmake {
    binary.addAll(listOf("lib/libopencv_core.so","lib/libopencv_highgui.so","libopencv_imgcodecs.so"  ))
}

cmake {
    binary.addAll(listOf("path/to/lib/directory1","path/to/lib/directory2"  ))
}

Current Behavior

does not allow adding a list of directories or .so/.a files. therefore the publishing plugin can not publish all produced libraries.

Script compilation error:

Line 29: binary.addAll(listOf("lib/libopencv_core.so","lib/libopencv_highgui.so","libopencv_imgcodecs.so" )) ^ Unresolved reference. None of the following candidates is applicable because of receiver type mismatch: public fun MutableCollection<in TypeVariable(T)>.addAll(elements: Array<out TypeVariable(T)>): Boolean defined in kotlin.collections public fun MutableCollection<in TypeVariable(T)>.addAll(elements: Iterable<TypeVariable(T)>): Boolean defined in kotlin.collections public fun MutableCollection<in TypeVariable(T)>.addAll(elements: Sequence<TypeVariable(T)>): Boolean defined in kotlin.collections

1 error

Context

Trying to build opencv and pushing it to the local maven repository.

Steps to Reproduce (for bugs)

Download . extract. copy inside gradle project. place build.gradle.kts inside extracted opencv:

plugins {
    id("org.gradle.samples.cmake-library") version "1.0"
    id ("org.gradle.samples.custom-publication") version "1.0"
}

group = "org.gradle.sample"
version = "3.4.6"

cmake {
    binary.addAll(listOf("lib/libopencv_core.so","lib/libopencv_highgui.so","libopencv_imgcodecs.so"  ))
    includeDirectory.set(layout.projectDirectory.dir("include"))
}

settings.gradle.kts: include (":opencv") ./gradlew publishToMavenLocal

Your Environment

Build scan URL: https://scans.gradle.com/s/va6p274ef22ic

@lacasseio any leads on this?

@lacasseio I described this in our Slack discussion. Here are some more details.

I have the same situation building Boost Libraries. Although in the case of Boost for some subset of the libraries there is a definitive separation of libraries + headers. Ideally I do not need to break the header files that come as a compete set to make subsets per Boost library feature. To make this worse Boost has the concept of header only libraries as many of the provided functionality is completely C++ template based and has no footprint in the system as a linkable shared or static library.

By Boost distribution own admission not all the included libraries have an easily observable boundary in the /usr/include/boost space either. So, making this work is going to be a chore for sure.

Before Gradle multi-variant publishing was available I created a plugin that I have been using to service these libraries that have one set of headers and multiple feature specific shared/static libraries. My plugin is a Software Model plugin which presented to the Software Model a model -> repositories -> myRepository structure populated dynamically with PrebuiltLibrary definitions. Each of these was informed by an additional metadata file published with the Boost packages allowing the extracted libraries to be usable as direct references in the BinarySpec.lib calls. The header only portion of Boost was represented by PrebuiltLibrary objects that did not define a link or runtime component but only a header component.

I hope to come up with a similar solution for this library in a new plugin or perhaps take the time to create a publishing model where the Boost library build generates a set of sub-projects based on a list of individual libraries each feeding from a common "installed" location which produces the defined libraries as header API ZIP files and matching shared/static libraries. Where there is only headers in a library the publication would list just the API ZIP main publication skipping the debug/release variants as there are none.

My previous solution combined the packaging and a consumer driven plugin to make this possible, I would like to try to make this fit the new model as much as possible.

@lacasseio one more detail I uncovered relating to cmake and other library tools with Linux library builds of shared libraries.

The Boost library generates "versioned" shared libraries. This is quite typical and involves embedding the versioned library name in the .so files. It is called "SONAME" in the Elf library format. Additionally, with a multi-module library like this often one library has a link to another one also expressed using the versioned library SONAME values. Take for example this Boost library:

$ patchelf --print-soname libboost_log.so.1.71.0 
libboost_log.so.1.71.0

$ patchelf --print-needed libboost_log.so.1.71.0
libboost_filesystem.so.1.71.0
libboost_thread.so.1.71.0
librt.so.1
libstdc++.so.6
libgcc_s.so.1
libpthread.so.0
libc.so.6
ld-linux-x86-64.so.2

Here you can see that the library has its SONAME as the versioned file and also other libboost component libraries are referenced as needed by their full versioned names. Publishing these as unversioned library files will not work as they no longer can function when installed, their needed links are not visible. When using a renamed library such as this the linker will actually use the SONAME inside to populate the NEEDED section of the library that depends on Boost so that too will fail to find the libraries after the linker completes.

When installed the Boost library shared files create the ldconfig traditional symlink structure like this:

$ ls -l libboost_log.so*
lrwxrwxrwx 1 avolanis avolanis       22 Jun 28 21:59 libboost_log.so -> libboost_log.so.1.71.0
lrwxrwxrwx 1 avolanis avolanis       22 Jun 28 21:59 libboost_log.so.1 -> libboost_log.so.1.71.0
lrwxrwxrwx 1 avolanis avolanis       22 Jun 28 21:59 libboost_log.so.1.71 -> libboost_log.so.1.71.0
-rwxr-xr-x 1 avolanis avolanis 21086944 Jun 30 10:18 libboost_log.so.1.71.0

One solution to this that I am trying now is to use the patchelf tool to change the SONAME and all NEEDED library references to the plain unversioned names in each Boost library. The syntax is pretty simple and may work in this case.

$patchelf --set-soname libboost_log.so libboost_log.so
$patchelf --replace-needed libboost_filesystem.so.1.71.0 libboost_filesystem.so libboost_log.so
...

While this might work I was wondering if publishing the versioned names would also be supported by Gradle publishing and what it might look like.

gradle / gradle-native