liblsl versioning on linux (soname and deb packages)

[tobiasherzke]

We are distributing 2 different open-source software packages that can make use of liblsl. For Linux, we distribute both software packages in our self-hosted apt repository, and we distribute liblsl deb packages as a dependency. Recently we ran into an issue: One of our software packages updated its dependency on liblsl from version 1.13 to version 1.14 while the other stayed at version 1.13. This led to a situation where we could not install both of our software packages simultaneously on the same machine.

We have investigated the issue a bit and want to suggest changes to the debian packaging of liblsl (if you are interested. If not, please feel free to close this issue).

Liblsl uses the full liblsl version to name the "soname" of the liblsl shared library on Linux. This has the following consequences if I compile a simple program on Linux that uses liblsl. I will first give a simple example:

#include <cstdio>
#include <lsl_cpp.h>
int main(int, char*[]) {
  printf("%s\n", lsl::library_info());
  return 0;
}

When I compile this program on Linux and link against liblsl with g++ c.cpp -llsl while liblsl 1.13.0 is installed, it will link against liblsl version 1.13.0 exactly:

$ ldd a.out 
    ...
    liblsl.so.1.13.0 => /lib/liblsl.so.1.13.0 (0x00
    ...

because

$ objdump -p /usr/lib/liblsl.so | grep SONAME
  SONAME               liblsl.so.1.13.0
$ ls -l /usr/lib/liblsl*
lrwxrwxrwx 1 root root      16 Nov 25  2020 /usr/lib/liblsl.so -> liblsl.so.1.13.0
-rw-r--r-- 1 root root 1273776 Nov 25  2020 /usr/lib/liblsl.so.1.13.0

You indicate with the full-version soname that there is no binary compatibility between any two liblsl versions: If I install an executable compiled against liblsl 1.13.0 and then upgrade the liblsl deb package to version 1.13.1, then my executable compiled against liblsl 1.13.0 can no longer execute. It wants to load liblsl .so.1.13.0, but this file is no longer available. Instead, file liblsl.so.1.13.1 is available, but the different "soname" indicates that there is no compatibility between 1.13.0 and 1.13.1.

I can imagine two different ways for you to make it easier for your users so that existing binaries continue to work after updating liblsl:

1) You may decide that the signified incompatibility between any two liblsl versions is unintended. You would have to tell your cmake build and packaging process a "soname" different from the full liblsl version for this to work.

2) You may decide that you do not want to deal with binary compatibility between liblsl versions, but provide a way for your users to have multiple verisons of liblsl installed simultaneously on Linux.

I can help investigating both options, but before that you should decide if the current situation is as intended for you, or if you want to fix it in one of the two suggested ways.

[tstenner]

As always, many thanks for the detailed report. :+1:

There was one backwards-incompatible ABI change that was needed to fix 64 bit types on Windows platforms, but so far no major change is planned. We should absolutely communicate this via the SONAME. As of now, the SONAME is simple the library version and only the installed CMake config hints that it's backwards compatible. I think it's surprisingly easy to fix:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index f39b222e..1cfb4ebc 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -6,6 +6,9 @@ project (liblsl
        HOMEPAGE_URL "https://github.com/sccn/liblsl"
        )

+# API version, to be incremented on backwards-incompatible ABI changes
+set(LSL_ABI_VERSION 2)
+
 set(CMAKE_CXX_VISIBILITY_PRESET hidden)
@@ -239,6 +242,7 @@ target_include_directories(lsl INTERFACE
 )
 set_target_properties(lsl PROPERTIES
        VERSION ${liblsl_VERSION_MAJOR}.${liblsl_VERSION_MINOR}.${liblsl_VERSION_PATCH}
+       SOVERSION ${LSL_ABI_VERSION}
 )

At least on my machine, it just works:

$ ldd lslver 
        linux-vdso.so.1 (0x00007ffee7b90000)
        liblsl.so.2 => ~/lsl/LSL/liblsl/build/liblsl.so.2 (0x00007f17ff84e000)
[..]

Having multiple versions installed side by side should be possible, but I suspect most users will resort to rpath shenanigans or use a wrapper for one the the other languages (i.e.) python that loads libraries from somewhere else. I hope I didn't overlook anything, but just to be sure @cboulay should approve of this change.

[tstenner]

For reference: SOVERSION

[tobiasherzke]

I can confirm that your proposed change to the CMakefile solves the problem nicely. Thank you!

[cboulay]

As long as the archive still has a liblsl.so that is sym-linked to the latest concrete liblsl.X.Y.so then the language wrappers should be fine.

[tobiasherzke]

There is:

# ls -l /usr/lib/liblsl*
lrwxrwxrwx 1 root root     11 Sep 27 12:02 /usr/lib/liblsl.so -> liblsl.so.2
-rw-r--r-- 1 root root 935552 Sep 27 12:02 /usr/lib/liblsl.so.1.15.2
lrwxrwxrwx 1 root root     16 Sep 27 12:02 /usr/lib/liblsl.so.2 -> liblsl.so.1.15.2

[cboulay]

Great! @tobiasherzke do you want to put together a PR so we can trigger the CI and make sure it works on all target platforms? I'm not worried about Windows, but I am a bit worried about Mac.

[tobiasherzke]

I can do that, but the change that I would put in that PR is really @tstenner's. Tristan, do you want to file that PR yourself or do you want me to do that?

[cboulay]

I was just chatting with Tristan and he says to go ahead. Thank you for asking.