search

conda-forge / pynio-feedstock

A conda-smithy repository for pynio.
BSD 3-Clause "New" or "Revised" License
3 stars 14 forks source link

`pynio` feedstock incompatible with `ujson` feedstock #114

Closed cisaacstern closed 2 years ago

cisaacstern commented 2 years ago

Solution to issue cannot be found in the documentation.

Issue

This feedstock appears to be incompatible with https://github.com/conda-forge/ujson-feedstock, as evidenced by the fact that in an environment built with

$ conda env create --file ujson-pynio.yml

from the specification

# ujson-pynio.yml
name: ujson-pynio
channels:
  - conda-forge
dependencies:
  - python=3.8
  - pynio
  - ujson

running

(ujson-pynio) $ python3 -c "import Nio"

raises

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/miniconda3/envs/ujson-pynio/lib/python3.8/site-packages/PyNIO/Nio.py", line 83, in <module>
    from _nio import *
ImportError: dlopen(/miniconda3/envs/ujson-pynio/lib/python3.8/site-packages/PyNIO/_nio.cpython-38-darwin.so, 0x0002): Library not loaded: @rpath/libtbb.dylib
  Referenced from: /miniconda3/envs/ujson-pynio/lib/libtiledb.dylib
  Reason: tried: '/miniconda3/envs/ujson-pynio/lib/libtbb.dylib' (no such file), '/miniconda3/envs/ujson-pynio/lib/libtbb.dylib' (no such file), '/miniconda3/envs/ujson-pynio/lib/libtbb.dylib' (no such file), '/miniconda3/envs/ujson-pynio/lib/libtbb.dylib' (no such file), '/miniconda3/envs/ujson-pynio/lib/libtbb.dylib' (no such file), '/miniconda3/envs/ujson-pynio/lib/libtbb.dylib' (no such file), '/miniconda3/envs/ujson-pynio/lib/libtbb.dylib' (no such file), '/miniconda3/envs/ujson-pynio/lib/python3.8/site-packages/PyNIO/../../../libtbb.dylib' (no such file), '/miniconda3/envs/ujson-pynio/lib/libtbb.dylib' (no such file), '/miniconda3/envs/ujson-pynio/lib/libtbb.dylib' (no such file), '/miniconda3/envs/ujson-pynio/lib/libtbb.dylib' (no such file), '/miniconda3/envs/ujson-pynio/lib/python3.8/site-packages/PyNIO/../../../libtbb.dylib' (no such file), '/miniconda3/envs/ujson-pynio/bin/../lib/libtbb.dylib' (no such file), '/miniconda3/envs/ujson-pynio/bin/../lib/libtbb.dylib' (no such file), '/usr/local/lib/libtbb.dylib' (no such file), '/usr/lib/libtbb.dylib' (no such file)

Installed packages

# packages in environment at /miniconda3/envs/ujson-pynio:
#
# Name                    Version                   Build  Channel
boost-cpp                 1.72.0               hdf9ef73_0    conda-forge
bzip2                     1.0.8                h0d85af4_4    conda-forge
c-ares                    1.18.1               h0d85af4_0    conda-forge
ca-certificates           2021.10.8            h033912b_0    conda-forge
cairo                     1.16.0            hec6a9b0_1003    conda-forge
cfitsio                   3.470                h422484a_7    conda-forge
curl                      7.76.1               h06286d4_1    conda-forge
expat                     2.4.7                h96cf925_0    conda-forge
fontconfig                2.13.96              h676cef8_2    conda-forge
freetype                  2.10.4               h4cff582_1    conda-forge
freexl                    1.0.6                h0d85af4_0    conda-forge
g2clib                    1.6.0                h4e57d6e_9    conda-forge
geos                      3.8.1                h4a8c4bd_0    conda-forge
geotiff                   1.5.1               hd8796ba_10    conda-forge
gettext                   0.19.8.1          hd1a6beb_1008    conda-forge
giflib                    5.2.1                hbcb3906_2    conda-forge
glib                      2.70.2               hcf210ce_4    conda-forge
glib-tools                2.70.2               hcf210ce_4    conda-forge
hdf4                      4.2.15               hefd3b78_3    conda-forge
hdf5                      1.10.5          nompi_h34ad4e8_1111    conda-forge
hdfeos2                   2.20              h207910d_1000    conda-forge
hdfeos5                   5.1.16               hc5a5cdf_6    conda-forge
icu                       64.2                 h6de7cb9_1    conda-forge
jasper                    1.900.1           h636a363_1006    conda-forge
jpeg                      9e                   h0d85af4_0    conda-forge
json-c                    0.13.1            h575e443_1002    conda-forge
kealib                    1.4.13               h2324030_0    conda-forge
krb5                      1.17.2               h60d9502_0    conda-forge
libblas                   3.9.0                8_openblas    conda-forge
libcblas                  3.9.0                8_openblas    conda-forge
libcurl                   7.76.1               h8ef9fac_1    conda-forge
libcxx                    12.0.1               habf9029_1    conda-forge
libdap4                   3.20.6               h3e144a0_2    conda-forge
libedit                   3.1.20191231         h0678c8f_2    conda-forge
libev                     4.33                 haf1e3a3_1    conda-forge
libffi                    3.4.2                h0d85af4_5    conda-forge
libgdal                   3.0.4                ha92c1e4_6    conda-forge
libgfortran               4.0.0           7_5_0_h1a10cd1_23    conda-forge
libgfortran4              7.5.0               h1a10cd1_23    conda-forge
libglib                   2.70.2               hf1fb8c0_4    conda-forge
libiconv                  1.16                 haf1e3a3_0    conda-forge
libkml                    1.3.0             h88bc94a_1012    conda-forge
liblapack                 3.9.0                8_openblas    conda-forge
libnetcdf                 4.7.4           nompi_he461dc0_101    conda-forge
libnghttp2                1.47.0               h942079c_0    conda-forge
libopenblas               0.3.12          openmp_h63d9170_1    conda-forge
libpng                    1.6.37               h7cec526_2    conda-forge
libpq                     12.3                 h7049927_3    conda-forge
libspatialite             4.3.0a            h658e6c1_1038    conda-forge
libssh2                   1.10.0               h52ee1ee_2    conda-forge
libtiff                   4.2.0                h355d032_0    conda-forge
libuuid                   2.32.1            h35c211d_1000    conda-forge
libwebp-base              1.2.2                h0d85af4_1    conda-forge
libxml2                   2.9.10               h53d96d6_0    conda-forge
libzlib                   1.2.11            h9173be1_1013    conda-forge
llvm-openmp               13.0.1               hcb1a161_1    conda-forge
lz4-c                     1.9.2                hb1e8313_3    conda-forge
ncurses                   6.3                  he49afe7_0    conda-forge
numpy                     1.22.3           py38hd52c74b_0    conda-forge
openjpeg                  2.3.1                h6cbf5cd_3    conda-forge
openssl                   1.1.1l               h0d85af4_0    conda-forge
pcre                      8.45                 he49afe7_0    conda-forge
pip                       22.0.4             pyhd8ed1ab_0    conda-forge
pixman                    0.38.0            h01d97ff_1003    conda-forge
poppler                   0.67.0               h16886b5_8    conda-forge
poppler-data              0.4.11               hd8ed1ab_0    conda-forge
postgresql                12.3                 he852019_3    conda-forge
proj                      7.0.0                h45baca5_5    conda-forge
pynio                     1.5.5           py38h97f92f9_12    conda-forge
python                    3.8.12          h17280f6_3_cpython    conda-forge
python_abi                3.8                      2_cp38    conda-forge
readline                  8.1                  h05e3726_0    conda-forge
setuptools                60.9.3           py38h50d1736_0    conda-forge
sqlite                    3.37.0               h23a322b_0    conda-forge
tiledb                    1.7.7                h84aa2a7_2    conda-forge
tk                        8.6.12               h5dbffcc_0    conda-forge
tzcode                    2021e                h0d85af4_0    conda-forge
ujson                     5.1.0            py38ha048514_0    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xerces-c                  3.2.2             h8f8adb3_1004    conda-forge
xz                        5.2.5                haf1e3a3_1    conda-forge
zlib                      1.2.11            h9173be1_1013    conda-forge
zstd                      1.4.8                h62961b2_0    conda-forge

Environment info

active environment : ujson-pynio
    active env location : /miniconda3/envs/ujson-pynio
            shell level : 2
       user config file :/.condarc
 populated config files : /.condarc
          conda version : 4.11.0
    conda-build version : not installed
         python version : 3.9.5.final.0
       virtual packages : __osx=10.16=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /miniconda3  (writable)
      conda av data dir : /miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/osx-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/osx-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /miniconda3/pkgs
                        /.conda/pkgs
       envs directories : /miniconda3/envs
                          /.conda/envs
               platform : osx-64
             user-agent : conda/4.11.0 requests/2.27.1 CPython/3.9.5 Darwin/21.0.1 OSX/10.16
                UID:GID : 501:20
             netrc file : None
           offline mode : False
hmaarrfk commented 2 years ago

This feedstock seems to need a good refresh. There are quite a few migrations that are open.

If you are vested in this, I would suggest opening up a PR that moves the migrations forward, and attempt to refresh this recipe. let me know if you need any guidance on that front.

ocefpaf commented 2 years ago

BTW, this project is nor under development as far as I know and most maintainers here moved away to other projects. I would consider this package on life support and it will only work on a specific env with older versions of our packages. Unless someone is willing to help out refresh it.

cisaacstern commented 2 years ago

Thanks for these suggestions, @hmaarrfk.

Echoing @ocefpaf, my understanding is that this project is no longer maintained and therefore I am not personally vested in making this work at the moment. I had it (unused) in a CI env as a holdover from an earlier development cycle, but ultimately removed it from that env when I discovered the conflict with ujson (which I do need in that env).

I raised this issue in case it ends up helping others who encounter a similar situation. (Apologies for the lack of clarity regarding this intention in my initial comment.)

ocefpaf commented 2 years ago

No problem. Let's keep this open and hope someone can pick up the maintenance here. Even though the package is deprecated there are still tons of workflows out there that requires it. An updated package with the latest migrations merged would be nice.

fulminemizzega commented 2 years ago

Hello, back in january I spent a couple of days trying to get pynio up to date, my changes have been included here: https://github.com/akrherz/pynio-feedstock/tree/rebuild I thought that then the changes would eventually end up also here. I do not use any of this stuff, but since I've already spent the time to get it to work, I'll try to do send here my changes, at least someone else can benefit from them. I've forked this feedstock again, migrated everything, rerendered and got it to compile on linux. I have no way to test what happens on mac OS with clang. I'll try to follow the correct procedure to make a pull request here and commit my changes. Right now, I can import Nio and ujson and nothing blows up. That said, I do not know if what you see here is related, libtbb is not a direct dependency of pynio or ujson (I think).

An updated package with the latest migrations merged would be nice.

Let's see if I can make one.

fulminemizzega commented 2 years ago

It is done. Unfortunately, all tests are failing because a server hosting a test file is down. Unfortunately.

fulminemizzega commented 2 years ago

@cisaacstern Now that the package has been refreshed, can you test if the issue is still there?

cisaacstern commented 2 years ago

🎉 @fulminemizzega awesome work! Confirming that the procedure described in https://github.com/conda-forge/pynio-feedstock/issues/114#issue-1166892339 no longer raises an error.

fulminemizzega commented 2 years ago

🎉 @fulminemizzega awesome work! Confirming that the procedure described in #114 (comment) no longer raises an error.

I did not do much... By sheer luck I noticed that hdf5 has an API compatibility macro-"monster" thing. The fix just boils down to a compile time setting.