Do not use pkg-config for libcrypto when a custom OpenSSL directory is specified

[itollefsen]

When installing OpenSSL (using make install), it's possible to specify a INSTALL_PREFIX (this can also be set during configuration). This is similar to the customary DESTDIR mechanism, but not quite.

DESTDIR will not duplicate the entire --prefix hierarchy, but OpenSSL's INSTALL_PREFIX will..

Given a package "foo" configured with configure --prefix=/opt/local:

• The customary make DESTDIR=/tmp/foo install will install it to /tmp/foo/bin, /tmp/foo/lib, etc.
• OpenSSL's make INSTALL_PREFIX=/tmp/foo install will install it to /tmp/foo/opt/local/bin, /tmp/foo/opt/local/lib, etc.

Package config files are generated based on the provided --prefix. So even if installed to somewhere else, the package config file will still point to the final destination provided during configuration.

This configure script would call pkg-config, if available, also when --with-openssl-dir was used.

There are old and pedantic reasons for that, primarily relating to finding the correct dependencies and avoiding specifying the library multiple times (event though that doesn't hurt).

However, if the mentioned INSTALL_PREFIX mechanism was used to install OpenSSL, pkg-config would add flags pointing to the --prefix location used when building OpenSSL, not the provided --with-openssl-dir.

Since the crypto library is so ubiquitous, this would most likely end up using the system library instead, if the rest of the configure checks found the functions it's looking for in the system library.

If there happened to be a crypto library already at the given --prefix, that would be used (again provided the configure checks passed).

Both of those scenarios are probably not what you'd expect to happen.

To fix this, this removes the use of pkg-config when --with-openssl-dir is specified.

To further make sure that --with-openssl-dir functions as the override it was meant to be, the check for a lib directory under the specified location is changed to a more comprehensive check for the library itself.

It uses the find utility to look for *crypto.*, which should find it for most naming schemes (it will find libcrypto.a, libcrypto.so, crypto.dll, etc.).

The find syntax used should be POSIX compliant. It has been tested with GNU, BSD and Solaris versions of find.

Since OpenSSL dependencies were handled by pkg-config, those have to now be added unconditionally when --with-openssl-dir is specified (in addition to when pkg-config isn't available).

This may result in a extraneous zlib (if OpenSSL was built without it) appearing on the link line (ref. "pedantic reasons" above). Modern linkers will not record a dependency on this unless explicitly told to do so.

[KevinWMatthews]

Interesting issue.

I am a bit confused, however. Doesn't GNU make's DESTDIR simply prepend to the installation path, just like OpenSSL's INSTALL_PREFIX? The make docs imply that this is the case:

/usr/local/lib/libfoo.a

can be installed as

/tmp/stage/usr/local/lib/libfoo.a

by setting DESTDIR=/tmp/stage.

Typically the path /usr/local/lib can be overridden using the --prefix option to an Autotools configure script.

Similarly, OpenSSL's INSTALL_PREFIX seems to have been created for a similar purpose (see here).

Perhaps I'm missing something or there's a variation between systems, but if this wasn't the case it seems like both package creation and cross-compilation would be very difficult. Thoughts?

On a side note, OpenSSL seems to replace INSTALL_PREFIX with DESTDIR in version OpenSSL_1_1_0. Presumably the variables behave in the same way?

[itollefsen]

I am a bit confused, however. Doesn't GNU make's DESTDIR simply prepend to the installation path, just like OpenSSL's INSTALL_PREFIX? The make docs imply that this is the case:

/usr/local/lib/libfoo.a

can be installed as

/tmp/stage/usr/local/lib/libfoo.a

by setting DESTDIR=/tmp/stage.

Only if --bindir and --libdir were fully specified (as /usr/local/[bin|lib]).

If you look at the output from configure --help, bindir is EPREFIX/bin. Which internally means $(exec_prefix)/bin, and the default value of exec_prefix is $(prefix). Same goes for libdir.

DESTDIR replaces $(prefix) and therefore $(exec_prefix). So unless you're overriding the various --*dir locations with absolute paths, you'll also get relative paths within DESTDIR.

These variables are described just below the chapter you linked (16.5: Variables for Installation Directories).

On a side note, OpenSSL seems to replace INSTALL_PREFIX with DESTDIR in version OpenSSL_1_1_0. Presumably the variables behave in the same way?

It looks like that's a straight up renaming job.

OpenSSL does $(DESTDIR)$(INSTALLTOP)/<foo>. If you translate that to Make-speak, it would be $(DESTDIR)$(prefix)/<foo>. But Make doesn't do that. It replaces the value of $(prefix) with the value of (DESTDIR), so you only have $(DESTIR)/<foo>.

[itollefsen]

I have another take on this laying around. Closing this and will post another one later.