[master/0.30.16] Statc build (-Dstatic=true) fails with link error: attempted static link of dynamic object `libserd-0.so.0.31.0'

hartwork commented 1 year ago

Hi, happy new year!

This was brought to my attention and I just verified the issue locally. Here's how to reproduce:

# cd "$(mktemp -d)"

# git clone --depth 1 https://github.com/drobilla/serd
Cloning into 'serd'...
remote: Enumerating objects: 1145, done.
remote: Counting objects: 100% (1145/1145), done.
remote: Compressing objects: 100% (760/760), done.
remote: Total 1145 (delta 239), reused 1027 (delta 219), pack-reused 0
Receiving objects: 100% (1145/1145), 424.59 KiB | 3.27 MiB/s, done.
Resolving deltas: 100% (239/239), done.

# cd serd/

# meson -Dstatic=true ./build && ninja -C ./build
The Meson build system
Version: 1.0.0
Source dir: /tmp/tmp.TS6RWN6Ff5/serd
Build dir: /tmp/tmp.TS6RWN6Ff5/serd/build
Build type: native build
Project name: serd
Project version: 0.31.0
C compiler for the host machine: cc (gcc 11.3.1 "cc (Gentoo 11.3.1_p20221223 p5) 11.3.1 20221223")
C linker for the host machine: cc ld.bfd 2.39
Host machine cpu family: x86_64
Host machine cpu: x86_64
Library m found: YES
Program doxygen found: YES (/usr/bin/doxygen)
Program sphinxygen found: NO
Program sphinx-build found: YES (/usr/bin/sphinx-build)
Program mandoc found: NO
Downloading sphinxygen source from https://download.drobilla.net/sphinxygen-1.0.0.tar.gz
Downloading: 100%|█████████████████████████████████████| 13894/13894 [00:00<00:00, 101804.17bytes/s]

Executing subproject sphinxygen 

sphinxygen| Project name: sphinxygen
sphinxygen| Project version: 0.0.1
sphinxygen| Program python3 (argparse, textwrap, xml.etree.ElementTree) found: YES (/usr/bin/python3) modules: argparse, textwrap, xml.etree.ElementTree
sphinxygen| Program sphinxygen found: YES (/tmp/tmp.TS6RWN6Ff5/serd/subprojects/sphinxygen-1.0.0/src/sphinxygen/sphinxygen.py)
sphinxygen| Build targets in project: 10
sphinxygen| Subproject sphinxygen finished.

Program sphinxygen found: YES (overridden)
Program python3 (sphinx_lv2_theme) found: NO
doc/c/meson.build:13: WARNING: Missing sphinx_lv2_theme module, falling back to alabaster
Configuring conf.py using configuration
Program doxygen found: YES (/usr/bin/doxygen)
Configuring Doxyfile using configuration
Program stylelint found: NO
Build targets in project: 14
NOTICE: Future-deprecated features used:
 * 0.64.0: {'copy arg in configure_file'}

serd 0.31.0

    API Documentation: YES
    Tests            : YES
    Tools            : YES
    Install prefix   : /usr/local
    Headers          : /usr/local/include
    Libraries        : /usr/local/lib64
    Executables      : /usr/local/bin
    Man pages        : /usr/local/share/man

  Subprojects
    sphinxygen       : YES

  User defined options
    static           : true

Found ninja-1.11.1 at /usr/bin/ninja
WARNING: Running the setup command as `meson [options]` instead of `meson setup [options]` is ambiguous and deprecated.
ninja: Entering directory `./build'
[24/34] Linking target serdi
FAILED: serdi 
cc  -o serdi serdi.p/src_serdi.c.o -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 '-Wl,-rpath,$ORIGIN/' -Wl,-rpath-link,/tmp/tmp.TS6RWN6Ff5/serd/build/ -Wl,--start-group libserd-0.so.0.31.0 -Wl,--end-group -static
/usr/lib/gcc/x86_64-pc-linux-gnu/11/../../../../x86_64-pc-linux-gnu/bin/ld: attempted static link of dynamic object `libserd-0.so.0.31.0'
collect2: error: ld returned 1 exit status
[29/34] Generating doc/c/singlehtml with a custom command
ninja: build stopped: subcommand failed.

Any ideas for a fix?

Thanks and best, Sebastian

eli-schwartz commented 1 year ago

It seems like you're trying to build a shared libserd (the default type of library), but a static tool? Why?

What's the purpose of this Gentoo use flag, actually?

hartwork commented 1 year ago

Hi Eli,

I had no intentions of making this about Gentoo but it's hard to answer your question without doing a bit of that:

The purpose is to enable or disable whether you want .a files in addition to .so files. The official definition is this:

# euse -i -g static-libs
global use flags (searching: static-libs)
************************************************************
[+ C    ] static-libs - Build static versions of dynamic libraries as well

For some package that means building things twice with different configuration, for some a single build can produce both, depends on the build system.

I had a closer look at meson.build now and learned that it takes -Dstatic=true -Ddefault_library=static to build a static library successfully.

To me that means that:

-Dstatic=true without -Ddefault_library=static breaks the build and can be argued a bug. It would be nice to either be caught and rejected with a helpful message or to turn those two flags into a single one. What do you think?
The Gentoo package will need to build twice for USE=static-libs or drop the flag.

Dropping the flag seems to be an option since no reverse depedency seems depends on dev-libs/serd[static-libs]:

# equery depends --all dev-libs/serd 2>/dev/null
* These packages depend on dev-libs/serd:
dev-libs/sord-0.16.14 (dev-libs/serd)
dev-libs/sord-9999 (dev-libs/serd)
media-libs/lilv-0.24.20 (dev-libs/serd[abi_x86_32(-)?,abi_x86_64(-)?,abi_x86_x32(-)?,abi_mips_n32(-)?,abi_mips_n64(-)?,abi_mips_o32(-)?,abi_s390_32(-)?,abi_s390_64(-)?])
media-libs/sratom-0.6.14 (dev-libs/serd)
media-sound/audacity-2.4.2-r3 (lv2 ? dev-libs/serd)
media-sound/jalv-1.6.8 (dev-libs/serd)
media-sound/sonic-visualiser-4.5 (dev-libs/serd)

PS: The ebuild is at https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-libs/serd/serd-0.30.16.ebuild if you're curious.

hartwork commented 1 year ago

PS: Fixed for Gentoo in https://github.com/gentoo/gentoo/commit/cabb7ede1fdc7cfe708352a07f863099ea847bae now.

eli-schwartz commented 1 year ago

The purpose is to enable or disable whether you want .a files in addition to .so files. The official definition is this:

Yes, I understand the general utility of this and I agree it has its uses. However, this isn't serd-specific, it's a fundamental meson option, so the static-libs USE flag should be setting default_library. And that USE flag could probably be part of the meson eclass, because as a fundamental meson option it's guaranteed to have the same semantics for all Meson projects, much like --prefix and --libdir work.

learned that it takes -Dstatic=true -Ddefault_library=static to build a static library successfully.

Nope, you just need that to link the tool executables statically. ;)

-Dstatic=true without -Ddefault_library=static breaks the build and can be argued a bug. It would be nice to either be caught and rejected with a helpful message or to turn those two flags into a single one. What do you think?

Yes, this is an unfortunate interaction and could probably be described as a bug.

But you can't combine both flags into a single one, because:

default_libary is a meson core option that controls building static libraries
static is a serd-specific option controlling whether to link statically or shared to the system libc when producing executables.

In order to link statically to the system libc, you really need to statically link to everything including libserd itself. So it makes no sense to do that unless you're generating static libraries too.

To fix the option clash, if this is interesting to the serd project, I'd probably recommend the following:

if static=true but default_library=shared, error out with the following error message: "cannot build static executables unless static libraries are also built"
if static=true and default_library=both, define serd_static_dep as a copy of serd_dep, except that link_with: libserd becomes libserd.get_static_lib()
else, use serd_dep as is (libserd will be a static library)

eli-schwartz commented 1 year ago

Either way, for correctness purposes, if Gentoo does not remove the USE flag, then the USE flag should either:

set default_library, or
be renamed to "static-bins" with a description "Build program binaries statically instead of linking to the system libc"

(My personal understanding is that there's interest in static libs but not static bins, so the latter option isn't the right choice to make.)

eli-schwartz commented 1 year ago

I took a quick look around the gentoo tree, it seems like other Meson-based packages with a static-libs USE flag pass this to emesonargs:

-Ddefault_library=$(usex static-libs both shared)

drobilla commented 1 year ago

Thanks @eli-schwartz. Indeed, the standard built-in default_library option used by every other meson project is what controls the type of library that's built.

As for "fixing" this situation, nah. Seems like the static option is doing exactly what it says it does to me, i.e., PEBKAC. A distribution clearly shouldn't be using it at all, because a distribution including both a shared library and static executables that use that library doesn't make any sense.

Conveniently, the compiler already has an error message for this situation, so no need to implement a redundant one in every meson script. People who feel compelled to tinker with options they don't understand will always be able to break the build somehow.

hartwork commented 1 year ago

Conveniently, the compiler already has an error message for this situation, so no need to implement a redundant one in every meson script. People who feel compelled to tinker with options they don't understand will always be able to break the build somehow.

@drobilla I was about to make a pull request to help serd users with the less than self-explaining error situation but okay. I'd rather help users understand the options better, catch invalid combinations and report about them in plain English. There is a lot of space between catching everything and catching things that will otherwise have users ask for help or file reports, like this ticket. Playing with things one doesn't understand is a way to learn, and warm water is more fun to learn swimming in than cold. Just my 2 cents.

learned that it takes -Dstatic=true -Ddefault_library=static to build a static library successfully.

Nope, you just need that to link the tool executables statically. ;)

@eli-schwartz I found --default-library=both working now, I guess that's why you noped about it.

I took a quick look around the gentoo tree, it seems like other Meson-based packages with a static-libs USE flag pass this to emesonargs:
-Ddefault_library=$(usex static-libs both shared)

@eli-schwartz interesting find, thank you :+1:

drobilla commented 1 year ago

Well, sorry if you got confused, but the option seems accurately described to me.

I maintain probably a dozen projects that use meson and contain both libraries and executables. Fixing this "issue" in serd wouldn't really do anything about this situation, and that's just for the projects maintained by one developer. It's simply not feasible to maintain a policy of friendly error messages for everything users can possibly do in every meson script in the world.

That said, it's sometimes a good idea, but in this case, it seems like meson itself has all of the information necessary to do so, even though this is a custom option (because it knows that the build is attempting to link a static binary to a shared executable), so serd or any other project isn't the right place to do this. Feel free to drive such an improvement on meson itself if you like, which would actually address this situation elegantly for everyone.

It would probably be better for meson to specifically support building static binaries where possible anyway, in which case it could handle it even better.

drobilla / serd

[master/0.30.16] Statc build (-Dstatic=true) fails with link error: attempted static link of dynamic object `libserd-0.so.0.31.0' #40