cmake builds an invalid library on Darwin

davidnich commented 7 years ago

the darwin library name when built with cmake is libqore.0.8.13.dylib

however the library name should be libqore.5.dylib

omusil24 commented 7 years ago

It seems that this happens on Linux too: libqore.so.0.8.13

But IMO, this is right... The numbers should be according to the version. I still don't understand why it should be 5.13.0 or something (the current situation when using autotools). Do you have any article that would explain this?

davidnich commented 7 years ago

@omusil24 the fact is that autotools have been the default standard build system, and therefore the autotools build set the standard. The version numbering was set in the autotools configuration a long time ago. Therefore if we change the build system, we have to maintain continuity with the old standard builds. Therefore we can't change the library versioning which by the way breaks every binary linked with the old version built by autotools. So from this point of view, the cmake version is not correct.

niclasr commented 7 years ago

One problem with trying to use libtools versioning on libraries is that it creates different library versions on different platforms. https://github.com/lxde/lxqt/issues/488#issuecomment-238084222 So in order to get the exact same behavior as libtool you need a macro that emulates this.

pvanek commented 7 years ago

also, from my perspective, libqore.so.0.8.13 makes much more sense than that one with cryptic 5 inside. It's a question how long we want to keep "wrong" legacy and how much we want to improve. I believe there is no standalone app using libqore which is not under our maintenance. Meaning

rpm/deb packages - all dependent packages are rebuilt when libqore changes
macports - the same (I hope)
qorus server - we are providing always full releases

Personally I don't care if it's 5.5.12 or 0.8.13, but the language version = libqore version seems more logical to me

davidnich commented 7 years ago

@pvanek this new naming convention makes sense from a certain point of view, but the old way also makes sense - and in fact if you understand ABI versioning, the old way makes much more technical sense than the "library file name version should match the release version" approach. The old way basically shows the ABI version information directly in the library name - it's a standard used by almost all UNIX software and there is definitely some sense behind it.

Equally importantly, because it wasn't the standard before, changing to "0.8.13" with the cmake build simply breaks things - meaning you have to recompile everything that links with libqore (meaning also modules that also linked with qore will no longer load for example - a qorus binary suddenly won't work because you rebuilt qore with cmake etc).

Using words like "right" and "wrong" and declaring one way "right" and other other "wrong" is a good way to make it a religious issue instead of talking about the technical merits of the situation.

pvanek commented 7 years ago

but how is possible that we have 5th version of ABI then? If I look into my rpm system it's almost always ABI version = library major version. That's all.

Yes, if we will change soname, we have to rebuild all linked apps. But it's not a showstopper for me. We control the whole stack.

davidnich commented 7 years ago

@pvanek the major number is 5 due to:

libqore_la_LDFLAGS = -version-info 19:0:14 -no-undefined ${QORE_LIB_LDFLAGS}

This means that the ABI is actually version 19 but it's backwards-compatible with the last 14 ABI releases, which results in major version 5.

Also the link provided by @niclasr above while explaining that cmake has difficulties in matching autotools' behavior also provides strong arguments for using the ABI version in the library name.

pvanek commented 7 years ago

that;s what I want to find the reason of it - how we can change ABI version without Qore's version? It's confusing (for me)

davidnich commented 7 years ago

that;s what I want to find the reason of it - how we can change ABI version without Qore's version? It's confusing (for me)

you can't, but that's not what this is about - this is about introducing an incompatible library naming convention against maintaining the status quo & keeping a standard, which is not absolute and not fully consistent across all platforms, but is a logical one at least in that the ABI is reflected in the name.

pvanek commented 7 years ago

hmm, regarding this

lib/Makefile.am:libqore_la_LDFLAGS = -version-info 19:0:14 -no-undefined ${QORE_LIB_LDFLAGS}

the ABI stuff is created manually. Also see the line's history - it looks like it's updated only "randomly". I'm afraid it's driven by "something fails" in depending apps only. I'm just guessing. As far as I understand there is no real tool to guard these ABI changes.

git log -L14,14:lib/Makefile.am

Regarding this we can solve this by simple adding:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 33213a0d9..dcb546efd 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -610,6 +610,7 @@ endif (WIN32 AND MINGW AND MSYS)

 set_target_properties(libqore PROPERTIES OUTPUT_NAME qore)
 set_target_properties(libqore PROPERTIES VERSION ${VERSION})
+set_target_properties(libqore PROPERTIES SOVERSION 5.13.0)
 if (APPLE)
     set_target_properties(libqore PROPERTIES INSTALL_NAME_DIR ${CMAKE_INSTALL_FULL_LIBDIR})
 endif (APPLE)

resulting into this on Linux:

pvanek@lister:~/src/qore/qore/build> ls -l lib*
lrwxrwxrwx 1 pvanek users      17 Jun 30 21:04 libqore.so -> libqore.so.5.13.0
-rwxr-xr-x 1 pvanek users 4975632 Jun 30 21:04 libqore.so.0.8.13
lrwxrwxrwx 1 pvanek users      17 Jun 30 21:04 libqore.so.5.13.0 -> libqore.so.0.8.13

I cannot test it in OSX now, as Roura's machine is off in the office.

davidnich commented 7 years ago

it didn't work on OSX - the results have both names somehow:

david@quasar-w:~/src/qore/git/qore/build$ ls -l libqore*
-rwxr-xr-x 1 david staff 4390076 Jun 30 21:30 libqore.0.8.13.dylib*
lrwxr-xr-x 1 david staff      20 Jun 30 21:30 libqore.5.13.0.dylib -> libqore.0.8.13.dylib*
lrwxr-xr-x 1 david staff      20 Jun 30 21:30 libqore.dylib -> libqore.5.13.0.dylib*

this way worked for me - but it's ugly:

set_target_properties(libqore PROPERTIES OUTPUT_NAME qore)
if (APPLE)
    set_target_properties(libqore PROPERTIES VERSION 5)
    set_target_properties(libqore PROPERTIES INSTALL_NAME_DIR ${CMAKE_INSTALL_FULL_LIBDIR})
else (APPLE)
    set_target_properties(libqore PROPERTIES VERSION 5.13.0)
endif (APPLE)

because on OSX the library is generated as libqore.5.dylib - only the major number is reflected in the name (another autotools inconsistency)

davidnich commented 7 years ago

however, it's more complicated - the name looks the same on OSX, but the cmake version results in the following:

libqore.dylib:
    /Users/david/src/qore/git/qore/build/libqore.5.dylib (compatibility version 0.0.0, current version 5.0.0)

whereas the autotools version looks like this:

david@quasar-w:~/src/qore/git/qore/build$ otool -L ../lib/.libs/libqore.dylib 
../lib/.libs/libqore.dylib:
    /usr/local/lib/libqore.5.dylib (compatibility version 20.0.0, current version 20.0.0)

Also for develop the autotools version string is 5.14.0; 5.13.0 is for 0.8.12...

davidnich commented 7 years ago

from some google searches it appears that:

autotools (libtool) doesn't handle library versioning correctly on BSD platforms (which includes OSX)
libtool's -version-info link flag is not supported by cmake; instead if you want compatibility between cmake and libtool, you have to use libtools old/deprecated -version-number link flag, which means that anyway we cannot provide the same ABI information with cmake as with libtool on platforms where libtool is not broken (Linux I suppose at least - Solaris not sure)

davidnich commented 7 years ago

so on Linux with the above solution we get with cmake:

david@el7:~/src/qore/git/qore/build$ objdump -p libqore.so.5.14.0 | grep SONAME
  SONAME               libqore.so.5.14.0

and with autotools:

david@el7:~/src/qore/git/qore/lib$ objdump -p /usr/lib64/libqore.so.5.14.0  | grep SONAME
  SONAME               libqore.so.5

Therefore the solution is I believe:

set_target_properties(libqore PROPERTIES OUTPUT_NAME qore)
if (APPLE)
    set_target_properties(libqore PROPERTIES VERSION 5)
    set_target_properties(libqore PROPERTIES SOVERSION 5)
    set_target_properties(libqore PROPERTIES INSTALL_NAME_DIR ${CMAKE_INSTALL_FULL_LIBDIR})
else (APPLE)
    set_target_properties(libqore PROPERTIES SOVERSION 5)
    set_target_properties(libqore PROPERTIES VERSION 5.14.0)
    set_target_properties(libqore PROPERTIES SOVERSION 5)
endif (APPLE)

which results in a cmake build on Linux:

david@el7:~/src/qore/git/qore/build$ ls -l libqore*
lrwxrwxrwx 1 david david      12 Jul  1 07:06 libqore.so -> libqore.so.5*
lrwxrwxrwx 1 david david      17 Jul  1 07:06 libqore.so.5 -> libqore.so.5.14.0*
-rwxrwxr-x 1 david david 4926840 Jul  1 07:06 libqore.so.5.14.0*
david@el7:~/src/qore/git/qore/build$ objdump -p libqore.so.5.14.0 | grep SONAME
  SONAME               libqore.so.5

and also a compatible library name on OSX:

david@quasar-w:~/src/qore/git/qore/build$ ls -l libqore.*;otool -L libqore.dylib 
-rwxr-xr-x 1 david staff 4390076 Jul  1 07:11 libqore.5.dylib*
lrwxr-xr-x 1 david staff      15 Jul  1 07:11 libqore.dylib -> libqore.5.dylib*
libqore.dylib:
    /Users/david/src/qore/git/qore/build/libqore.5.dylib (compatibility version 5.0.0, current version 5.0.0)

because anyway versioning was broken on OSX in any case; every minor change of the library resulted in a new version & compatibility version, however the name is the same and linking between the libtool and cmake version of the library is interchangeable despite the internal version difference (5 vs 20).

niclasr commented 7 years ago

According to https://cmake.org/cmake/help/v3.8/prop_tgt/SOVERSION.html version corresponds to the current version and SOVERSION to the compatible version on OSX. So what does setting VERSION to 20 and SOVERSION to 5 result in?

davidnich commented 7 years ago

@niclasr this leads to the following result:

david@quasar-w:~/src/qore/git/qore/build$ ls -l libqore.*;otool -L libqore.dylib 
-rwxr-xr-x 1 david staff 4390076 Jul  1 18:50 libqore.20.dylib*
lrwxr-xr-x 1 david staff      16 Jul  1 18:50 libqore.5.dylib -> libqore.20.dylib*
lrwxr-xr-x 1 david staff      15 Jul  1 18:50 libqore.dylib -> libqore.5.dylib*
libqore.dylib:
    /Users/david/src/qore/git/qore/build/libqore.5.dylib (compatibility version 5.0.0, current version 20.0.0)

qorelanguage / qore

cmake builds an invalid library on Darwin #1988