ctypes find_library() on osx

[kalefranz]

From @rokuingh on June 25, 2015 19:25

When used on osx, the ctypes find_library() command does not find dependency libraries (probably because it does not search anaconda lib directories - seems to work fine on linux). This can be worked around by setting DYLD_LIBRARY_PATH. A similar issue was discussed in ticket #308, with the final decision being that the underlying issues in the library (ZMQ::FFI) should be fixed rather than depending on DYLD_LIBRARY_PATH. What is the suggested solution for building conda distributions on osx for libraries that depend on ctypes.find_library()?

Copied from original issue: conda/conda#1394

[kalefranz]

From @asmeurer on June 25, 2015 19:28

I've seen this issue before. We may need to patch find_library so that it does the right thing in Anaconda.

[kalefranz]

From @rokuingh on June 25, 2015 21:15

We've seen some similar issues in other libraries which use ctypes. Will work on tracking those back to their source as well, if you like. Do you have a handle on the patch to find_library()? Let me know if there is anything my team can do to help.

On 6/25/15 1:28 PM, Aaron Meurer wrote:

I've seen this issue before https://github.com/ioos/conda-recipes/issues/184. We may need to patch find_library so that it does the right thing in Anaconda.

— Reply to this email directly or view it on GitHub https://github.com/conda/conda/issues/1394#issuecomment-115370125.

[kalefranz]

From @rokuingh on June 25, 2015 21:16

Sorry about that last mail, I meant to say that we've seen some similar issues in libraries that use Cython!

On 6/25/15 3:15 PM, Ryan OKuinghttons wrote:

We've seen some similar issues in other libraries which use ctypes.
Will work on tracking those back to their source as well, if you like. Do you have a handle on the patch to find_library()? Let me know if there is anything my team can do to help.

On 6/25/15 1:28 PM, Aaron Meurer wrote:

I've seen this issue before https://github.com/ioos/conda-recipes/issues/184. We may need to patch find_library so that it does the right thing in Anaconda.

— Reply to this email directly or view it on GitHub https://github.com/conda/conda/issues/1394#issuecomment-115370125.

[kalefranz]

From @richlewis42 on July 7, 2016 10:1

Any developments on this? cairocffi uses find_library to find cairo's dylibs, but can't find those from the conda cairo package on Mac, as these are put in the anaconda libs dir - it works when you add the lib dir to DYLD_LIBRARY_PATH, but from reading above, this is clearly not the way to fix this.

[kalefranz]

From @rokuingh on July 8, 2016 2:18

Not much development here, although there is another developer in our group @bekozi working on a possibly related issue with Darwin systems not correctly linking the installed version of a library. It is currently being handled using install_name_tool : https://sourceforge.net/p/esmf/tix/3613956/

[kalefranz]

From @basnijholt on February 3, 2017 0:52

We may need to patch find_library so that it does the right thing in Anaconda.

This would be great! Is it somehow on a list somewhere?

[kalefranz]

CC @mingwandroid

[kalefranz]

From @rolando on May 8, 2017 15:4

I had this issue with ijson (requires yajl lib from the conda-forge channel).

In [1]: import ijson.backends.yajl2
---------------------------------------------------------------------------
YAJLImportError                           Traceback (most recent call last)
<ipython-input-1-cad02275774a> in <module>()
----> 1 import ijson.backends.yajl2

/Users/rolando/miniconda3/envs/py35/lib/python3.5/site-packages/ijson/backends/yajl2.py in <module>()
     10
     11
---> 12 yajl = backends.find_yajl_ctypes(2)
     13
     14 yajl.yajl_alloc.restype = POINTER(c_char)

/Users/rolando/miniconda3/envs/py35/lib/python3.5/site-packages/ijson/backends/__init__.py in find_yajl_ctypes(required)
     25     so_name = util.find_library('yajl')
     26     if so_name is None:
---> 27         raise YAJLImportError('YAJL shared object not found.')
     28     yajl = cdll.LoadLibrary(so_name)
     29     require_version(yajl.yajl_version(), required)

YAJLImportError: YAJL shared object not found.

In [5]: import os

In [6]: os.environ['DYLD_FALLBACK_LIBRARY_PATH'] = '/Users/rolando/miniconda3/envs/py35/lib/'

In [7]: import ijson.backends.yajl2

In [8]:

[kalefranz]

Moving this to anaconda-issues. The proposal here is to patch python find_library, so it'll have to go through that issue tracker anyway.

[ukaratay]

@kalefranz any updates on this?

[mingwandroid]

I have patched find_library so that it does the right thing for AD.

Please help to test the packages.

conda update -c c3i_test2 python

For those who are interested, my patch is here

[jakirkham]

Thanks for the pointer @mingwandroid. Have opened an issue ( https://github.com/conda-forge/python-feedstock/issues/157 ) to note that we should do the same in conda-forge. Don't have time to do this myself ATM, but would happily review PRs for the supported Python versions in conda-forge and get them in if anyone is interested.

[mingwandroid]

Getting these patches into conda-forge should be part of the general mass-dump when conda-forge adopts conda-build 3 and our new compilers. I'm happy to make the PRs the minute that happens.

[jakirkham]

Sorry @mingwandroid. Should have broken that comment into two parts. Was just trying to generally point out where it could be done if someone interested in resolving this issue wanted to make this happen.

[KnightOfTheMoonlight]

for a little reference for someone encounter this issue again. I have solved it by ln -s /opt/homebrew/lib/libgs.dylib /Users/lilin/opt/anaconda3/bin/../lib/libgs.dylib

[jakirkham]

@KnightOfTheMoonlight this is a rather old issues with patches since applied to the python packages from defaults & conda-forge. If you are still seeing an issue that looks like this, it is quite likely a different issue.

Would recommend raising a new issue with all the details needed to reproduce it from scratch. Can't say exactly where this should be opened based on the info provided, but may be able to provide some guidance on that with more details.