Incompatible libjpeg library

[duncanwp]

Doing a clean install of these dependencies using the condo-forge channel gives incompatible libraries:

(cis_dev)aopposxlap18:cis_repo_test_files watson-parris$ python -m'netCDF4' Traceback (most recent call last): File "/Users/watson-parris/anaconda/envs/cis_dev/lib/python2.7/runpy.py", line 163, in _run_module_as_main mod_name, _Error) File "/Users/watson-parris/anaconda/envs/cis_dev/lib/python2.7/runpy.py", line 111, in _get_module_details import(mod_name) # Do not catch exceptions initialising package File "/Users/watson-parris/anaconda/envs/cis_dev/lib/python2.7/site-packages/netCDF4/init.py", line 3, in from ._netCDF4 import * ImportError: dlopen(/Users/watson-parris/anaconda/envs/cis_dev/lib/python2.7/site-packages/netCDF4/_netCDF4.so, 2): Library not loaded: @rpath/libjpeg.8.dylib Referenced from: /Users/watson-parris/anaconda/envs/cis_dev/lib/libnetcdf.11.dylib Reason: Incompatible library version: libnetcdf.11.dylib requires version 13.0.0 or later, but libjpeg.8.dylib provides version 12.0.0

These are the libraries conda picks up:

(cis_dev)aopposxlap18:cis_repo_test_files watson-parris$ conda list

packages in environment at /Users/watson-parris/anaconda/envs/cis_dev:

biggus 0.14.0 py27_1 condo-forge blas 1.1 openblas conda-forge ca-certificates 2016.2.28 0 condo-forge cartopy 0.14.2 np111py27_1 condo-forge cf-units 1.1.1 cf_units 1.1.1 py27_0 conda-forge curl 7.49.1 0 condo-forge cycler 0.10.0 py27_0 defaults ecmwf_grib 1.15.0 0 condo-forge expat 2.1.0 1 condo-forge freetype 2.6.3 1 condo-forge funcsigs 1.0.2 py27_0 defaults functools32 3.2.3.2 py27_1 conda-forge geos 3.5.0 0 defaults hdf4 4.2.11 4 condo-forge hdf5 1.8.17 2 condo-forge icu 56.1 4 condo-forge iris 1.9.2 py27_1 conda-forge jasper 1.900.1 2 https://conda.binstar.org/scitools/osx-64/jasper-1.900.1-2.tar.bz2 jpeg 9b 0 condo-forge libgfortran 3.0.0 0 condo-forge libiconv 1.14 3 condo-forge libmo_unpack 3.0 1 condo-forge libnetcdf 4.4.1 0 defaults libpng 1.6.23 0 condo-forge libtiff 4.0.6 6 condo-forge libxml2 2.9.3 9 condo-forge libxslt 1.1.28 2 lxml 3.6.1 py27_0 defaults matplotlib 1.5.2 np111py27_4 condo-forge mo-pack 0.2.0 mo_pack 0.2.0 np111py27_0 condo-forge mock 2.0.0 py27_0 defaults netcdf4 1.2.4 np111py27_2 condo-forge nose 1.3.7 py27_1 defaults numpy 1.11.1 py27_blas_openblas_201 [blas_openblas] condo-forge openblas 0.2.18 4 condo-forge openssl 1.0.2h 1 defaults owslib 0.11.0 py27_0 conda-forge pandas 0.18.1 np111py27_1 condo-forge pbr 1.10.0 py27_0 defaults pillow 3.3.0 py27_0 condo-forge pip 8.1.2 py27_0 defaults proj.4 4.9.2 0 condo-forge psutil 4.3.0 py27_0 conda-forge pyepsg 0.3.1 py27_0 conda-forge pyhamcrest 1.8.3 py27_0 file:///Users/watson-parris/anaconda/conda-bld/osx-64/pyhamcrest-1.8.3-py27_0.tar.bz2 pyhdf 0.9.0 np111py27_1 condo-forge pyke 1.1.1 py27_0 https://conda.binstar.org/scitools/osx-64/pyke-1.1.1-py27_0.tar.bz2 pyparsing 2.1.4 py27_0 defaults pyproj 1.9.5.1 py27_0 conda-forge pyshp 1.2.3 py27_0 scitools python 2.7.12 1 defaults python-dateutil 2.5.3 py27_0 defaults python-ecmwf_grib 1.15.0 np111py27_0 condo-forge pytz 2016.6.1 py27_0 defaults readline 6.2 2 requests 2.10.0 py27_0 defaults scipy 0.18.0 np111py27_blas_openblas_201 [blas_openblas] condo-forge setuptools 23.0.0 py27_0 defaults shapely 1.5.16 py27_0 defaults six 1.10.0 py27_0 defaults sqlite 3.13.0 0 defaults tk 8.5.18 0 udunits2 2.2.20 1 condo-forge wheel 0.29.0 py27_0 defaults xz 5.2.2 0 condo-forge zlib 1.2.8 3 defaults

The weird thing is that I have libjpeg 9 installed, so for some reason it looks like the netcdf install isn't linking against it?

[ocefpaf]

I see a mixture of channels there that worries my. One is the Scitools channel and the other is the weird netcdf4 1.2.4 np111py27_2 condo-forge. That should be conda-forge :confused: (Maybe it is just an alias from when you added the channel... Not sure.)

Can you issue the following commands for me so we can debug this further:

conda create -n TEST -c conda-forge netcdf4 --yes
source activate TEST
python -c "import netCDF4; print(netCDF4.__version__)"

and if that does not work send us the results of:

conda list  > debug.txt
conda info >> debug.txt

Thanks!

---

Edit: BTW I tried with your requirements file and the .condarc below and everything worked fine on Linux. It is either an issue on OS X or a bad mix of channels. The results above will help us sort this out.

> cat ~/.condarc 
channels:
    - conda-forge
    - defaults
show_channel_urls: true

[duncanwp]

Great, thanks. OK so the TEST environment worked fine - it picked up version 1.1.1 and linked against the OS libjpeg 8d.

Yes, it only seems to be a problem on OSX, and I'm wondering if it's because of the built-in libjpeg library?

This is the result with the above .condarc file: debug.txt. It's still picking up some sci-tools packages - I must have them installed in my root env - but I don't think there's anything there that should mess with the NetCDF dependency resolution.

[ocefpaf]

but I don't think there's anything there that should mess with the NetCDF dependency resolution.

The dependency tree might play tricks with us and it is hard to follow it sometimes, but jasper depends on jpeg and scitools does not pin the jpeg version. Which means that you have some packages, like hdf4 which is a dependency of netcdf4, compiled against jpeg 9* and conda is play loose with that pin because other packages you are installing are not pinned.

In theory you do not need the scitools channel at all when using conda-forge b/c everything that exists there also exists in conda-forge. I strongly recommend you to reduce the number of channels, the more you mix the more conflicts like that you will get.

Since the problem is not in conda-forge I am closing this, but we can keep discussing this until we solve your problem.

[duncanwp]

Perfect, thanks for that - I was struggling to understand why conda wasn't picking the right library. I've built a clean environment with only condo-forge libraries and all is working as expected.

Thanks again

[ocefpaf]

Glad it is resolved!