OpenCV depends on old hdf5 version, breaking pytables

luhk commented 5 years ago

Actual Behavior

Using conda install pytables, hdf5 1.10.4 and pytables 3.5.2 get installed. Afterwards, conda install opencv downgrades hdf5 to 1.10.2 and in turn pytables to 3.4.4, which is incompatible with numpy 1.16 and therefore breaks pandas; see PyTables/Pytables#717.

Expected Behavior

opencv does not depend on the patch level of the hdf5 version number.

Steps to Reproduce

conda install pandas

Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: xxx

  added / updated specs:
    - pandas

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    six-1.12.0                 |           py37_0          23 KB
    ------------------------------------------------------------
                                           Total:          23 KB

The following NEW packages will be INSTALLED:

  pandas             pkgs/main/linux-64::pandas-0.24.2-py37he6710b0_0
  python-dateutil    pkgs/main/linux-64::python-dateutil-2.8.0-py37_0
  pytz               pkgs/main/noarch::pytz-2019.1-py_0
  six                pkgs/main/linux-64::six-1.12.0-py37_0

Proceed ([y]/n)? y

Downloading and Extracting Packages
six-1.12.0           | 23 KB     | ######################################################### | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

conda install pytables

Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: xxx

  added / updated specs:
    - pytables

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    lz4-c-1.8.1.2              |       h14c3975_0         130 KB
    lzo-2.10                   |       h49e0be7_2         182 KB
    zstd-1.3.7                 |       h0b5b093_0         401 KB
    ------------------------------------------------------------
                                           Total:         712 KB

The following NEW packages will be INSTALLED:

  blosc              pkgs/main/linux-64::blosc-1.16.3-hd408876_0
  bzip2              pkgs/main/linux-64::bzip2-1.0.8-h7b6447c_0
  hdf5               pkgs/main/linux-64::hdf5-1.10.4-hb1b8bf9_0
  lz4-c              pkgs/main/linux-64::lz4-c-1.8.1.2-h14c3975_0
  lzo                pkgs/main/linux-64::lzo-2.10-h49e0be7_2
  mock               pkgs/main/linux-64::mock-3.0.5-py37_0
  numexpr            pkgs/main/linux-64::numexpr-2.6.9-py37h2ffa06c_0
  pytables           pkgs/main/linux-64::pytables-3.5.2-py37h71ec239_1
  snappy             pkgs/main/linux-64::snappy-1.1.7-hbae5bb6_3
  zstd               pkgs/main/linux-64::zstd-1.3.7-h0b5b093_0

Proceed ([y]/n)? 

Downloading and Extracting Packages
lzo-2.10             | 182 KB    | ######################################################### | 100% 
zstd-1.3.7           | 401 KB    | ######################################################### | 100% 
lz4-c-1.8.1.2        | 130 KB    | ######################################################### | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

conda install opencv

Collecting package metadata (current_repodata.json): done
Solving environment: failed with current_repodata.json, will retry with next repodata source.
Initial quick solve with frozen env failed.  Unfreezing env and trying again.
Solving environment: failed with current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: xxx

  added / updated specs:
    - opencv

The following NEW packages will be INSTALLED:

  cairo              pkgs/main/linux-64::cairo-1.14.12-h8948797_3
  ffmpeg             pkgs/main/linux-64::ffmpeg-4.0-hcdf2ecd_0
  fontconfig         pkgs/main/linux-64::fontconfig-2.13.0-h9420a91_0
  freeglut           pkgs/main/linux-64::freeglut-3.0.0-hf484d3e_5
  freetype           pkgs/main/linux-64::freetype-2.9.1-h8a8886c_1
  glib               pkgs/main/linux-64::glib-2.56.2-hd408876_0
  graphite2          pkgs/main/linux-64::graphite2-1.3.13-h23475e2_0
  harfbuzz           pkgs/main/linux-64::harfbuzz-1.8.8-hffaf4a1_0
  icu                pkgs/main/linux-64::icu-58.2-h9c2bf20_1
  jasper             pkgs/main/linux-64::jasper-2.0.14-h07fcdf6_1
  jpeg               pkgs/main/linux-64::jpeg-9b-h024ee3a_2
  libglu             pkgs/main/linux-64::libglu-9.0.0-hf484d3e_1
  libopencv          pkgs/main/linux-64::libopencv-3.4.2-hb342d67_1
  libopus            pkgs/main/linux-64::libopus-1.3-h7b6447c_0
  libpng             pkgs/main/linux-64::libpng-1.6.37-hbc83047_0
  libtiff            pkgs/main/linux-64::libtiff-4.0.10-h2733197_2
  libuuid            pkgs/main/linux-64::libuuid-1.0.3-h1bed415_2
  libvpx             pkgs/main/linux-64::libvpx-1.7.0-h439df22_0
  libxcb             pkgs/main/linux-64::libxcb-1.13-h1bed415_1
  libxml2            pkgs/main/linux-64::libxml2-2.9.9-hea5a465_1
  opencv             pkgs/main/linux-64::opencv-3.4.2-py37h6fd60c2_1
  pcre               pkgs/main/linux-64::pcre-8.43-he6710b0_0
  pixman             pkgs/main/linux-64::pixman-0.38.0-h7b6447c_0
  py-opencv          pkgs/main/linux-64::py-opencv-3.4.2-py37hb342d67_1

The following packages will be DOWNGRADED:

  hdf5                                    1.10.4-hb1b8bf9_0 --> 1.10.2-hba1933b_1
  pytables                             3.5.2-py37h71ec239_1 --> 3.4.4-py37ha205bf6_0

Proceed ([y]/n)? n

CondaSystemExit: Exiting.

Anaconda or Miniconda version:

Operating System:

`conda info`

``` active environment : hdftest active env location : /home/xxx/anaconda3/envs/hdftest shell level : 2 user config file : /home/xxx/.condarc populated config files : /home/xxx/.condarc conda version : 4.7.10 conda-build version : 3.18.9 python version : 3.7.3.final.0 virtual packages : base environment : /home/xxx/anaconda3 (writable) channel URLs : https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /home/xxx/anaconda3/pkgs /home/xxx/.conda/pkgs envs directories : /home/xxx/anaconda3/envs /home/xxx/.conda/envs platform : linux-64 user-agent : conda/4.7.10 requests/2.22.0 CPython/3.7.3 Linux/4.15.0-55-generic neon/18 .04 glibc/2.27 UID:GID : 1000:1000 netrc file : None offline mode : False ```

`conda list --show-channel-urls`

``` # packages in environment at /home/xxx/anaconda3/envs/hdftest: # # Name Version Build Channel _libgcc_mutex 0.1 main defaults blas 1.0 openblas defaults blosc 1.16.3 hd408876_0 defaults bzip2 1.0.8 h7b6447c_0 defaults ca-certificates 2019.5.15 0 defaults certifi 2019.6.16 py37_1 defaults hdf5 1.10.4 hb1b8bf9_0 defaults libedit 3.1.20181209 hc058e9b_0 defaults libffi 3.2.1 hd88cf55_4 defaults libgcc-ng 9.1.0 hdf63c60_0 defaults libgfortran-ng 7.3.0 hdf63c60_0 defaults libopenblas 0.3.6 h5a2b251_1 defaults libstdcxx-ng 9.1.0 hdf63c60_0 defaults lz4-c 1.8.1.2 h14c3975_0 defaults lzo 2.10 h49e0be7_2 defaults mock 3.0.5 py37_0 defaults ncurses 6.1 he6710b0_1 defaults nomkl 3.0 0 defaults numexpr 2.6.9 py37h2ffa06c_0 defaults numpy 1.16.4 py37h99e49ec_0 defaults numpy-base 1.16.4 py37h2f8d375_0 defaults openssl 1.1.1c h7b6447c_1 defaults pandas 0.24.2 py37he6710b0_0 defaults pip 19.1.1 py37_0 defaults pytables 3.5.2 py37h71ec239_1 defaults python 3.7.3 h0371630_0 defaults python-dateutil 2.8.0 py37_0 defaults pytz 2019.1 py_0 defaults readline 7.0 h7b6447c_5 defaults setuptools 41.0.1 py37_0 defaults six 1.12.0 py37_0 defaults snappy 1.1.7 hbae5bb6_3 defaults sqlite 3.29.0 h7b6447c_0 defaults tk 8.6.8 hbc83047_0 defaults wheel 0.33.4 py37_0 defaults xz 5.2.4 h14c3975_4 defaults zlib 1.2.11 h7b6447c_3 defaults zstd 1.3.7 h0b5b093_0 defaults ```

luhk commented 5 years ago

Besides, OpenCV is now at version 4.1.1. Will there be an update?

benmwebb commented 4 years ago

As far as I can tell, this also prevents OpenCV from being installed at all with latest Anaconda, since OpenCV depends on HDF5 1.10.2 and Anaconda on 1.10.4:

UnsatisfiableError: The following specifications were found to be incompatible with each other:                                                                                                             

Package hdf5 conflicts for:
anaconda==2019.10 -> h5py==2.9.0=py37h3134771_0 -> hdf5[version='>=1.10.4,<1.10.5.0a0']
opencv=3.4.2 -> hdf5[version='>=1.10.2,<1.10.3.0a0']

benmwebb commented 4 years ago

Any plan for an update here? This also prevents any user of OpenCV from using Python 3.8.

benmwebb commented 4 years ago

I'm happy to help here, although it doesn't seem there's anything that needs doing, other than just rebuilding the package. I rebuilt what appears to be the current recipe from @mingwandroid and it picked up the new HDF5 without issues. With the resulting package at our channel our users can now install our conda packages again (which depend on opencv) with anaconda or Python 3.8. It was easy enough to build for Python 3.8 too with a quick /3.7m?/3.8/, /37m?/38/ replace (although obviously that's a quick hack).

This works for us but obviously it would be preferable if the package in base were up to date.

ContinuumIO / anaconda-issues