SainsburyWellcomeCentre / aeon_mecha

Project Aeon's main library for interfacing with acquired data. Contains modules for raw data file io, data querying, data processing, data qc, database ingestion, and building computational data pipelines.
BSD 3-Clause "New" or "Revised" License
6 stars 6 forks source link

Fix macOS wheel build for `datajoint` (Issue #249) #406

Closed MilagrosMarin closed 2 months ago

MilagrosMarin commented 2 months ago

Summary: This PR addresses issue #249, which involves a wheel build failure for datajoint on macOS 12 when using mamba, and proposes three solutions for macOS users.

Also, this PR:

Context: On macOS, when attempting to install datajoint through a conda/mamba environment using an .yml file (see below), the following error occurs during the installation:

name: aeon_test
channels:
  - conda-forge
  - defaults
dependencies:
  - python>=3.11
  - pip
  - blas>=2.0, <3
  - bottleneck>=1.2.1, <2
  - dash
  - dotmap
  - fastparquet
  - graphviz
  - ipykernel
  - jupyter
  - jupyterlab
  - matplotlib
  - numba>=0.46.0, <1
  - numexpr>=2.6.8, <3
  - numpy>=1.21.0, <2
  - pandas>=1.3
  - plotly
  - pyarrow
  - pydotplus
  - pymysql
  - pyyaml
  - scikit-learn
  - scipy
  - seaborn
  - xarray>=0.12.3, <1
  - pip:
      - datajoint>=0.13.6, <1
      - git+https://github.com/datajoint-company/datajoint-utilities.git
      - opencv-python
error: subprocess-exited-with-error
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [48 lines of output]
      /Users/milagros/miniconda/envs/aeon_test/lib/python3.12/site-packages/setuptools/__init__.py:88: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
      !!
              ********************************************************************************
              Requirements should be satisfied by a PEP 517 installer.
              If you are using pip, you can try `pip install --use-pep517`.
              ********************************************************************************

...

      OSError: [Errno 66] Directory not empty: '/private/var/folders/h8/_lx50k3d6qx586662kr066h40000gn/T/pip-install-ovdbidfz/datajoint_530cc0190d4342fc91f8abda5f5fdf76/.eggs/watchdog-5.0.2-py3.12-macosx-11.0-arm64.egg/watchdog-5.0.2.dist-info' -> '/private/var/folders/h8/_lx50k3d6qx586662kr066h40000gn/T/pip-install-ovdbidfz/datajoint_530cc0190d4342fc91f8abda5f5fdf76/.eggs/watchdog-5.0.2-py3.12-macosx-11.0-arm64.egg/EGG-INFO'
      [end of output]
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed cleaning build dir for datajoint
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (datajoint)
failed
CondaEnvException: Pip failed

setuptools no longer recommends using setup.py for package installation. Specifically, setuptools is now raising warnings that setup.py install is deprecated in favor of using modern standards-based tools such as PEP 517 to build packages.

This deprecation is affecting how datajoint package is resolved during the build process for Apple Silicon (M1/M2) Macs. The python packages listed in the setup.py of datajoint dependency are not properly installed.

Solution: As python evolves to modernize package building and installation, the transition from setup.py to pyproject.toml (following PEP 517 and PEP 518) will help standardize the process. To stay aligned with these best practices, datajoint-python can consider updating its dependency management to use pyproject.toml for smoother builds and installations for mac users.

Temporal solutions are:

Proposed solution in this PR for macOS installation process:

Tests: Locally using conda, and using GHA with mamba.

Environment details: macOS: 12, 14.5 python: 3.11, 3.12.6 conda/mamba: installation attempted via conda/mamba env create -f env.yml

Main packages:

setuptools                73.0.1
wheel                     0.44.0
pip                       24.2