Complications with HDF5 dependency on mac silicon

[jameshalgren]

This is a bit esoteric, but I'll document it here in case anyone else runs into it. Also note that the issue will probably resolve as the dependencies are updated.

The specific issue appears to be with the PyTables dependency, and specifically the HDF5 library within that dependency all when running on the Apple M1 chip. This StackOverflow issue summarizes things nicely: https://stackoverflow.com/questions/66147426/trying-to-install-pytables-for-python3

Ultimately the fix was to install pytables first, then to pip install hydrotools. The pytables installation requires explicit declaration of the hdf5 path:

brew install hdf5
HDF5_DIR=/Users/user/homebrew/opt/hdf5 pip install tables
pip install hydrotools

[jameshalgren]

The error (for reference) looked like this:

pip install tables
.
.
.
Collecting tables
  Using cached tables-3.7.0.tar.gz (8.2 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [12 lines of output]
      /var/folders/6q/k8j4psmn1nb2w5q19_scthth0000gq/T/H5close9u55vh5d.c:2:5: error: implicit declaration of function 'H5close' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
          H5close();
          ^
      1 error generated.
      cpuinfo failed, assuming no CPU features: No module named 'cpuinfo'
      * Using Python 3.8.9 (default, Apr 13 2022, 08:48:06)
      * Found cython 0.29.30
      * USE_PKGCONFIG: False
      .. ERROR:: Could not find a local HDF5 installation.
         You may need to explicitly state where your local HDF5 headers and
         library can be found by setting the ``HDF5_DIR`` environment
         variable or by using the ``--hdf5`` command-line option.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
.
.
.
%

[hellkite500]

How did you install HDF5? Or did you install it prior to trying to install this package?

[hellkite500]

FWIW this has been actively worked in PyTables/PyTables#832 and my guess is that resolving your installation of HDF5 will get through this.

[jarq6c]

@jameshalgren Thanks for reporting. Do you know if this issue also occurs when using conda or is it a pip specific problem?

[jarq6c]

If the problem is pip specific, a conda-forge package of hydrotools might help resolve the issue.

https://github.com/conda-forge/staged-recipes https://github.com/conda-incubator/grayskull

[hellkite500]

It looks like a good build of hdf5 has hit homebrew for arm. This error still creeps up, but can be resolved by installing hdf5 from homebrew

brew install hdf5

and then installing tables and explicitly pointing to the brew hdf5 dir via an environment variable

HDF5_DIR=<homebrew_location>/opt/hdf5 pip install tables

This should also work for installing tables as a dependency, e.g.

HDF5_DIR=<homebrew_location>/opt/hdf5 pip install hydrotools

[aaraney]

and then installing tables and explicitly pointing to the brew hdf5 dir via an environment variable

HDF5_DIR=<homebrew_location>/opt/hdf5 pip install tables

To simplify this a bit, the following is equivalent (ignoring the export):

export HDF5_DIR=$(readlink -f $(brew --prefix hdf5))