compiler.find_library() fallback inside dependency()

[roshbaby]

When a library is not included in the package manager but is nevertheless present in the build environment (a good example being zlib on darwin):

zlib_dep = dependency('zlib', required: false) if not zlib_dep.found() zlib_dep = meson.get_complier('c').find_library('z', required: false) endif

In theory, one could argue that the above pattern should be used for all (library) dependencies in a project.

Is it possible for this to be subsumed within the dependency() function by, perhaps, allowing the user to specify a list of compiler objects & the corresponding find_library() names to use as fallback? There might be better avenues I haven't thought of yet.

[jpakkane]

I seem to recall we had a thing for looking up zlib on macOS so it would return the system library automatically from dependency('zlib') but I can't seem to find it now...

[nirbheek]

Duplicate of https://github.com/mesonbuild/meson/issues/2654.

I don't think we can do this in a generic fashion for all libraries because the name is usually different.

[roshbaby]

I wasn't expecting meson to manage the name differences (similarly, I don't think meson should have added special handling for zlib internally).

My original request above was different - more along the lines of the fallback keyword in dependency(), e.g.

mylibdep = dependency('mylib', clib_fallback : ['c', 'my'])

where meson first looks for 'mylib' in the package manager, and if not found, looks for the library 'my' in the 'c' compiler object. Both strings 'mylib' and 'my' are provided by the user so that meson doesn't have to implement any registry of names.

In that I don't think that this is a Duplicate of 2654.

[nirbheek]

similarly, I don't think meson should have added special handling for zlib internally

This sort of special-casing inside meson is critical to keep build files simple, and to have projects buildable on a wide variety of platforms. If every project had to maintain a list of all possible names of zlib, half would get it wrong, and the other half would copy and cargo-cult it without ever testing that it actually works.

However, for other cases when people want to deal with distros not shipping pkg-config files for libraries, it might be useful to have a concise way of specifying the fallback library. Reopening.

[nirbheek]

In addition to a find_library(), we also need a way to specify which headers must be available.

[xclaesse]

I had the crazy idea of using dict to create a chain of dependencies:

libfoo_chain = [
  # Check for pkg-config name first
  'foo',
  # Check for an alternative pkg-config name
  'libfoo-1.0',
  # Check for a C library with headers
  {
    'language' : 'c',
    'library' : 'foo',
    'headers' : ['foo.h']
  },
  # Check for an alternative library name
  {
    'language' : 'c',
    'library' : 'libfoo-1.0',
    'headers' : ['foo.h']
  },
  # Fallback to a subproject
  {
    'subproject' : 'foo',
    'dependency' : 'libfoo_dep'
  },
]
libfoo_dep = dependecy(libfoo_chain, required : get_option('some_feature'))

[dcbaker]

I agree with @nirbheek, we shouldn't put this logic in the meson.build files this kind of logic belongs in core meson.

[xclaesse]

@dcbaker for some well-known libraries like zlib, meson should have built-in support indeed. But we still need the mechanism exposed to meson.build IMHO.

[dcbaker]

What libraries do you have in mind other than libraries like zlib that are provided as part of the base OS and don't have an alternative discovery method like pkg-config?

[nirbheek]

libpng, see this monstrosity: https://gitlab.gnome.org/GNOME/gdk-pixbuf/blob/master/meson.build#L234

cdparanoia, which refuses to live in 2018: https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/blob/master/ext/cdparanoia/meson.build#L8

And generally I see GNOME libraries littered with fallback code for linking with custom-built zlib/pcre/foo on Windows with Visual Studio, such as:

https://gitlab.gnome.org/GNOME/glib/blob/master/meson.build#L1715

https://gitlab.gnome.org/GNOME/gtk/blob/master/meson.build#L333

We should definitely add custom dependency classes for very common libraries like zlib or vulkan, or openssl (which has different library names on Windows!).

But it's harder to do the same for others like libpng which have different APIs on different versions, and only the consumer can know which versions they support, or for libraries that have unpredictable names.

[dcbaker]

libpng at least has the version in the name, so we could implement logic that does something like:

dependency('libpng', version : ['>= 10', '< 17'])

and know that you mean until ['libpng', 'libpng10', 'libpng11', ... 'libpng17'].

I'm not sure what to do about the windows stuff. Just wrap all of those things and fallback instead?

[bluca]

Another use case is for projects adding a pkg-config file in newer versions, but consumers needing to support older versions as well.

Case in point: libpcap finally added a pkg-config file upstream in 1.9.0, but most distros do not have it yet. So we have to do the awkward dance with dependency() first and fallback to find_library() later if we want to support any distro version shipped before 2019.

Any update on this implementation? Thanks!

[nirbheek]

For pcap, we already have a dependency class, so the fix there is to have it look for pkg-config first, like other dependency classes.

[bluca]

That would be great for pcap. However it's not the first project that adds a .pc file long after the initial release and I'm sure it won't be the last, so a generic fallback would help in many more cases IMHo

[nirbheek]

Yes, I agree.

[nirbheek]

Another example (that should probably be an openssl dependency class): https://gitlab.gnome.org/GNOME/glib-networking/blob/master/meson.build#L80

[dkrikun]

I got here after searching for half an hour how to link to a library on Windows. It is a common practice to have 3rd-party dependencies prebuilt at a potentially arbitrary location or under some 'foreign/' sub-directory of a project. Having to go through the wraps, subprojects etc. to be able to consume a 3rd-party on Windows is a huge burden..

[jpakkane]

You can add paths so search for with the dirs keyword argument. You can either use meson.current_source_dir() or something like:

lib_dep = cc.find_library('alexandria', dirs : get_option('thirdparty_lib_dir'))

Where thirdparty_lib_dir is a project option set in meson_options.txt.

[dkrikun]

@jpakkane Sorry for ignorance, I just tried this out, and:

1. It required me to specify full path (Search directory <xxx> is not an absolute path)
2. It didn't find anything -- no clue about what was the problem

As a windows (i.e. msvc) user this is not very clear -- would be great if we could just specify library search path the same way one can pass include_directories to e.g. executable target.

[Hi-Angel]

So, what's the recommended way of dealing with this right now? E.g. in case of whitedb we had to write this in main meson.build file:

WHITEDB            = dependency('whitedb', fallback : ['whitedb', 'whitedb_dep'])

And then we created a file subprojects/whitedb/meson.build with just 2 lines:

project('whitedb')
whitedb_dep = declare_dependency(link_args : '-lwgdb')

As I understand, it's far from ideal, because we are not really building the subproject, all we wanted is a fallback to the library if system has no whitedb.pc file. Is there a better way?

[xclaesse]

It's ugly, but you can do it like that:

dep = dependency('whitedb', required : false)
if not dep.found()
  dep = cc.find_library('wgdb', has_header : 'something.h')
endif

[louiz]

I don’t understand where the commit 967c1e404e7453d18d413d79c9d169defa875fb6 went… It looks like it’s supposed to make it possible to fallback to a find_library() in one single call, but I can’t find that in the doc, or even in the current code (in master).

That would have been ideal. Currently I have to do:

udns = dependency('udns', required: get_option('udns'))
if get_option('udns').disabled() == false and udns.found() == false
   udns = cpp.find_library('udns', has_headers: 'udns.h', required: get_option('udns'))                                                                                                                                                                                                                                                                                                                                                   
endif

and this is not great since it displays three messages, one red NO followed by two green YES. This is confusing for the users.

[xclaesse]

@louiz that commit is from a PR that never got merged. It needs more work. https://github.com/mesonbuild/meson/pull/4595