support for systems without pkg-config

[svigerske]

Issue created by migration from Trac.

Original creator: @svigerske

Original creation time: 2018-10-17 15:38:57

Assignee: @svigerske

Keywords: autotools-update

Currently we rely on pkg-config being available. That seems to be common on Linux and Mac OS X. Also with MSys2 it is either already there by default or easy to install. So maybe we can assume its presence? If not, we could think about distributing an own replacement, e.g., from https://github.com/pkgconf/pkgconf

[svigerske]

Comment by @LouHafer created at 2018-11-01 18:44:24

I'm in favour of assuming pkgconf or pkg-config will be present, in the absence of a compelling example of a system we need to support that doesn't have it. If a compelling example comes up, I vote for distributing a replacement, but that gets us back to ThirdParty software again. We definitely don't want to get back to reimplementing pkgconf in autotools macros.

[svigerske]

Comment by @svigerske created at 2018-11-01 18:49:32

I'm in favor of this for the moment, too.

I had put this in here to cleanup another todo list, but used priority "minor" and type "enhancement" to indicate that this would rather be something far down the road.

[svigerske]

Comment by @svigerske created at 2018-11-03 14:37:15

Input from Ted:

'' Pkg-config should be available on every system that we are supporting, but I see two issues. First is that pkg-config may often have to be installed manually, which adds complexity. I'm not sure if it comes by default on every Linux distro, but I don't think it comes by default on OS X (I could be wrong) and requiring users to get it via homebrew if that is the only reason they need to use homebrew might be a little nasty. On Windows, the problem is that there are multiple versions of pkg-config out there (windows paths versus posix) so things can get screwed up easily. We would have to experiment with what works well, but u think when building on Msys with Visual Studio, it may be impossible to get things right. ''

[tkralphs]

To start up this conversation again (and hopefully resolve it), I wanted to relay that a user reported to me that Osi configuration had failed on OS X because CoinUtils was not found. The issue was simply be that pkg-config was not installed, so the fix was to install it, simple. However, the error message did not (does not) indicate anything useful for solving the problem. Perhaps this should be opened as a separate issue, but should we provide a more useful message to indicate to users when a library is not found that they must either install pkg-config or supply --with-* arguments to indicate where to find the library and header files? Is there any other option?

We used to have a --with-coin-instdir option, which was eliminated. The purpose of that argument was just to say "look here for installed packages if all else fails." Seems to me that we could still do something like that in the case where pkg-config is not present. If we don't want to reintroduce --with-coin-instdir, we could still at least do the link check with -L$prefix/lib and -I $prefix/include/coin-or, couldn't we? Even if we don't have pkg-config, this would be enough to allow a coinbrew build to succeed without pkg-config.

So with respect to the original question, I would say we don't currently require the existence of pkg-config (right?) and there's really no reason to strictly do so. One can specify how to link to all required dependency libraries using --with-* arguments. However, to avoid a lot of pain, most users probably do want pkg-config and we should suggest to people that they get it if they don't have it. If we make the small modification that we check to see if we can find the libraries already installed in the prefix, though, pkg-config won't really be needed in many cases, which would be nice for removing a small amount of friction for some people.

Thoughts?

[svigerske]

To start up this conversation again (and hopefully resolve it), I wanted to relay that a user reported to me that Osi configuration had failed on OS X because CoinUtils was not found. The issue was simply be that pkg-config was not installed, so the fix was to install it, simple. However, the error message did not (does not) indicate anything useful for solving the problem. Perhaps this should be opened as a separate issue, but should we provide a more useful message to indicate to users when a library is not found that they must either install pkg-config or supply --with-* arguments to indicate where to find the library and header files? Is there any other option?

We could have a look whether the message that pkg-config is not available is prominent enough and consequences of not having it available are emphasized sufficiently.

We used to have a --with-coin-instdir option, which was eliminated. The purpose of that argument was just to say "look here for installed packages if all else fails." Seems to me that we could still do something like that in the case where pkg-config is not present. If we don't want to reintroduce --with-coin-instdir, we could still at least do the link check with -L$prefix/lib and -I $prefix/include/coin-or, couldn't we? Even if we don't have pkg-config, this would be enough to allow a coinbrew build to succeed without pkg-config.

I don't think it was "if all else fails". Wasn't it adding the coin-instdir to the beginning of the PKG_CONFIG_PATH? A user can just do that directly, there is no need to have an extra functionality in configure that just augments the pkg-config search path. We also already add $prefix to the front of PKG_CONFIG_PATH automatically, which is more useful now that $prefix isn't the build directory by default anymore.

If we were to introduce reasonable defaults for all the --with arguments in AC_COIN_CHK_PKG, then we would have to also add an actual link check there, too. So next to bringing back guessing some compiler and linker flags (which may easily fail with static builds), we also need to figure out what symbol to check for and which headers to include. With pkg-config we took the comfortable assumption that the existence of the .pc file (or more exactly a good return from pkg-config --exists) is enough to indicate that the package is present and can be used.

So with respect to the original question, I would say we don't currently require the existence of pkg-config (right?) and there's really no reason to strictly do so. One can specify how to link to all required dependency libraries using --with-* arguments. However, to avoid a lot of pain, most users probably do want pkg-config and we should suggest to people that they get it if they don't have it. If we make the small modification that we check to see if we can find the libraries already installed in the prefix, though, pkg-config won't really be needed in many cases, which would be nice for removing a small amount of friction for some people.

Just that it's not a "small modification". It brings back hardcoding linker and compiler flags of dependencies that I was happy to get rid of many years ago (though, back then, these were for building against uninstalled projects, which was more painful) and would require adding linkchecks for COIN-OR projects.

Thoughts?

[LouHafer]

I'm with Stefan here. It's not a small modification. Make the `Install pkg-config' message more prominent. Add a suggestion that the user check PKG_CONFIG_PATH to make sure that the Coin installation directory is at the front. We've just spent a few years dragging ourselves into the modern world. Our users can do the same.

[tkralphs]

First, I'm talking only about the situation when there is no pkg-config, so it's not about PKG_CONFIG_PATH. I agree with you that if it's just a matter of the PKG_CONFIG_PATH not being set right, we can expect the user to do that (although even in this case, we could output a nice message telling them this might be the issue).

I guess I see what you're saying now about this not being a small change, but it still seems rather "simple" from the standpoint of coinbrew. We can easily figure out the names of the dependent libraries because we know the dependencies and we know what the libraries associated with them are going to be called. So building default --with-* arguments is easy in coinbrew (in fact, I am already doing that for the purposes of linking to downloaded libraries in the Visual Studio case). But OK, it's probably harder to do as part of configuration and is something that belongs in coinbrew and not in coin.m4.

Given the current TODO list and that we haven't gotten very many reports of issues related to lack of pkg-config, I'll put this on the back burner. For now, I think we essentially assume the existence of pkg-config or that the user will specify --with-* arguments for all dependencies manually. I added a check to coinbrew to warn the user if pkg-config does not exist.

With all that said, when configure fails due to lack of pkg-config, it only bails out when it can't find a required package, saying something like "Required package CoinUtils not available." This is going to meaningless to most users and will be especially confusing if they can see that CoinUtils was just built and installed. If they were to look at the output, they would see that there was an earlier failure to find pkg-config, but I doubt they would see the connection between that and the failure to find the lackage, since the lack of pkg-config does not lead to any kind of error or warning. Since we can ascertain that (1) pkg-config was not found and (2) there were no --with-* arguments given, we could easily spit out a nice message explaining what happened and the options for fixing it. Of course, this is a bit mott now for users of coinbrew, since coinbrew already does this.

[svigerske]

Yeah, I think we all agreed that a longer message could help.

[svigerske]

The lack of pkg-config should now lead to a warning for every package that wasn't skipped (e.g., --without-pkg), where .pc wasn't skipped (AC_COIN_CHK_PKG(,,[skip])) and that could not be found.

[tkralphs]

Hmm, I don't think the import of that warning will be understood by naive users either. Can't we just say something more detailed, in plain English?

You do not seem to have pkg-config installed and have also not supplied paths to dependent 
libraries with `--with-pkg-lflags` and `--with-pkg-cflags`. Although you may have already built
and installed the dependencies, they may not be found. The easiest fix is to install pkg-config. 
See full build documentation at https://coin-or.github.io

Also, can we put it in stable/0.8, too?

[svigerske]

I made it a bit longer.

I'm skipping pointing to https://coin-or.github.io as this only applies to Optimization Suite projects, not necessarily everyone using BuildTools. is about a lot of stuff and not particular about "you need pkg-config". Saying "pkg-config is not found" and "pkg-config should be installed" should really be enough.

Also I'm not going to add things in stable/0.8 anymore. That one even has some hacks to simulate pkg-config if not available.

[tkralphs]

Ok, fair enough. I guess we can close this issue now?

[svigerske]

Fine with me, but I would leave that to you.

[tkralphs]

You open, you close :).