Cannot find libzmq in linux

[tsdev]

On my linux installation transplant cannot find the libzmq library. The location of the libzmq library is /usr/local/lib64/libzmq.so.5.1.3, however it is not searched by transplant:

          search_dirs = ((os.getenv('LD_LIBRARY_PATH') or '').split(':') +
                           self._read_ldsoconf('/etc/ld.so.conf') +
                           ['/lib/', '/lib64/',
                            '/usr/lib/', '/usr/lib64/'])

If you prefer, I can create a pull request for these kind of simple fixes, however you come up most often a better solution than mine. I also have a different libzmq version in /usr/local/lib so it might be good to search that folder as well.

[bastibe]

It should be searched as part of self._read_ldsoconf('/etc/ld.so.conf'). As far as I can decipher man ldconfig, all additional search paths should be saved somewhere in /etc/ld.so.conf, and in files included therein. If you know a better way of getting a list of all valid search paths, or the actual path of libzmq on Linux, I'd be grateful for any hints.

I suspect that this is a bug in self._read_ldsoconf, or yet another creative Linux configuration.

Can you check the output of _read_ldsoconf on your system, and show me the contents of your /etc/ld.so.conf (and included files)?

On an unrelated note, I will be on vacation until August 9th, and probably won't be able answer Github issues during that time.

[tsdev]

The file /etc/ld.so.conf contains:

include ld.so.conf.d/*.conf

The referenced conf files contain some library specific path only, for example atlas-x86_64.conf:

/usr/lib64/atlas

So nowhere /usr/local/lib64 can be found.

[tsdev]

The machine is running Centos 7 and zmq was installed with the default options.

[bastibe]

Does man ldconfig mention that path? Does ctypes.util.find_library find your libzmq? I wonder where it gets that path from?

I don't want to just include /usr/local/lib64 in the search string without knowing why it should be included. Maybe I should try to read /etc/ld.so.cache instead, but I don't know how to interpret that file's contents. Or maybe there is some command line utility that can just tell me the correct path.

[tsdev]

The man ldconfig mentions only the path that you have already. According to the documentation of ctypes.util.find_library it searches binary files (from /sbin/ldconfig and using gcc and objdump). And I just found the following command that returns exactly what we need:

ld --verbose | grep SEARCH_DIR | tr -s ' ;' \\012

When I run it on my system I get:

SEARCH_DIR("/usr/x86_64-redhat-linux/lib64")
SEARCH_DIR("/usr/lib64")
SEARCH_DIR("/usr/local/lib64")
SEARCH_DIR("/lib64")
SEARCH_DIR("/usr/x86_64-redhat-linux/lib")
SEARCH_DIR("/usr/local/lib")
SEARCH_DIR("/lib")
SEARCH_DIR("/usr/lib")

If you clean this, you will get the necessary libraries. Maybe using this:

ld --verbose | grep SEARCH_DIR | tr -s ' ;' \\012 | awk -F"\"" '{print $(NF-1)}'

[bastibe]

This is disgusting. My computer returns a similar set of paths, some of which are not mentioned anywhere else.

See 71aa8cd0cb4d251c613ef825131f9cdb1f3bf2e9 for an implementation that uses this information as well. Does this work for you?

[tsdev]

I am using pip now, are these changes automatically propagated so that I can just reinstall transplant with pip?

[bastibe]

No, these changes are not yet on pip. They are only available on Github, because I didn't have a chance to test them thoroughly, yet. In particular, I want to test this on a few more distros to make sure that it doesn't blow up if some distro uses a different version of ld or whatever.

That said, you can install the version from git by cloning the repo to your system, and pip install /path/to/cloned/transplant or python setup.py install. This will install the local copy, instead of pulling from PyPi.

[tsdev]

The update works as expected!