search

MITHaystack / digital_rf

Read, write, and interact with data in the Digital RF and Digital Metadata formats
Other
102 stars 31 forks source link

'hdf5.h' file not found during install on Apple Silicon #38

Open aidanmontare-edu opened 2 years ago

aidanmontare-edu commented 2 years ago

I am unable to install digital_rf on my M1 Mac. Has anyone else run into issues on Apple Silicon? (I am using Python directly on arm, not through Rosetta).

When I run pip install digital_rf, it fails with fatal error: 'hdf5.h' file not found.

The first time I tried this, it failed during the install of a dependency, h5py. I was able to get h5py installed and working by first installing hdf5 from homebrew and then providing this path when calling pip: HDF5_DIR=/opt/homebrew/opt/hdf5 pip install --no-binary=h5py h5py. I verified the h5py was installed correctly by running the tests as described in its documentation.

Even though hdf5 and h5py are now installed, installing digital_rf still fails with the same file not found error. The full output of pip install digital_rf is below. My best guess is that even though I have the dependencies somewhere on the system, it is not something that the install can find. However, I'm not well-versed in these things, so any help would be appreciated.

A few system details:

CPU: M1 Pro
Mac OS 12.3
python 3.10.2 (installed using pyenv)
pip 22.0.4 hdf5 1.13.0 (via homebrew) h5py 3.6.0

Collecting digital_rf
  Using cached digital_rf-2.6.7.tar.gz (1.2 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: packaging in ./.pyenv/versions/3.10.2/lib/python3.10/site-packages (from digital_rf) (21.3)
Requirement already satisfied: six in ./.pyenv/versions/3.10.2/lib/python3.10/site-packages (from digital_rf) (1.16.0)
Requirement already satisfied: python-dateutil in ./.pyenv/versions/3.10.2/lib/python3.10/site-packages (from digital_rf) (2.8.2)
Collecting watchdog
  Using cached watchdog-2.1.6-cp310-cp310-macosx_11_0_arm64.whl (86 kB)
Requirement already satisfied: h5py in ./.pyenv/versions/3.10.2/lib/python3.10/site-packages (from digital_rf) (3.6.0)
Requirement already satisfied: numpy in ./.pyenv/versions/3.10.2/lib/python3.10/site-packages (from digital_rf) (1.22.2)
Requirement already satisfied: pytz in ./.pyenv/versions/3.10.2/lib/python3.10/site-packages (from digital_rf) (2021.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in ./.pyenv/versions/3.10.2/lib/python3.10/site-packages (from packaging->digital_rf) (3.0.7)
Building wheels for collected packages: digital_rf
  Building wheel for digital_rf (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for digital_rf (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [37 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.macosx-12.2-arm64-3.10
      creating build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/ringbuffer.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/mirror.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/digital_rf_deprecated_hdf5.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/_version.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/util.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/drf_command.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/__init__.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/list_drf.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/watchdog_drf.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/digital_metadata.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/digital_rf_hdf5.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      creating build/lib.macosx-12.2-arm64-3.10/gr_digital_rf
      copying gr_digital_rf/vector.py -> build/lib.macosx-12.2-arm64-3.10/gr_digital_rf
      copying gr_digital_rf/__init__.py -> build/lib.macosx-12.2-arm64-3.10/gr_digital_rf
      copying gr_digital_rf/digital_rf_sink.py -> build/lib.macosx-12.2-arm64-3.10/gr_digital_rf
      copying gr_digital_rf/raster.py -> build/lib.macosx-12.2-arm64-3.10/gr_digital_rf
      copying gr_digital_rf/digital_rf_source.py -> build/lib.macosx-12.2-arm64-3.10/gr_digital_rf
      UPDATING build/lib.macosx-12.2-arm64-3.10/digital_rf/_version.py
      set build/lib.macosx-12.2-arm64-3.10/digital_rf/_version.py to '2.6.7'
      running build_ext
      INFO: pkgconfig cannot find HDF5. Defaulting to HDF5_ROOT="/Users/aidan/.pyenv/versions/3.10.2"
      building 'digital_rf._py_rf_write_hdf5' extension
      creating build/temp.macosx-12.2-arm64-3.10
      creating build/temp.macosx-12.2-arm64-3.10/lib
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -I/private/var/folders/7w/6t8zqxtj03n_hhb4jmx6knh00000gn/T/pip-install-2wrim69t/digital-rf_a3d1bf96baeb4f93a17ee50fbdaa775b/include -I/Users/aidan/.pyenv/versions/3.10.2/include/python3.10 -I/private/var/folders/7w/6t8zqxtj03n_hhb4jmx6knh00000gn/T/pip-build-env-atbhlnz4/overlay/lib/python3.10/site-packages/numpy/core/include -I/Users/aidan/.pyenv/versions/3.10.2/include -I/opt/local/include -I/usr/local/include -I/usr/include -c lib/py_rf_write_hdf5.c -o build/temp.macosx-12.2-arm64-3.10/lib/py_rf_write_hdf5.o
      In file included from lib/py_rf_write_hdf5.c:24:
      /private/var/folders/7w/6t8zqxtj03n_hhb4jmx6knh00000gn/T/pip-install-2wrim69t/digital-rf_a3d1bf96baeb4f93a17ee50fbdaa775b/include/digital_rf.h:46:10: fatal error: 'hdf5.h' file not found
      #include "hdf5.h"
               ^~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for digital_rf
Failed to build digital_rf
ERROR: Could not build wheels for digital_rf, which is required to install pyproject.toml-based projects
ryanvolz commented 2 years ago

@jswoboda Can you take a look at this on your M1?

@aidanmontare-edu Have you ever used the conda package manager before? You might have better luck installing digital_rf with that since it will already be compiled. At least until we can see about making the source build work better for the M1.

aidanmontare-edu commented 2 years ago

@ryanvolz I'm not currently a conda user, but that's something I could try

aidanmontare-edu commented 2 years ago

@ryanvolz I installed miniconda on my system and was able to install digital_rf! I ran one or two of the examples, which finished fine, so this seems like a success.

I'd still be interested in a way to install / build with pip, if that becomes possible down the road. I am happy to be a tester for work on this.