Project Aeon's main library for interfacing with acquired data. Contains modules for raw data file io, data querying, data processing, data qc, database ingestion, and building computational data pipelines.
BSD 3-Clause "New" or "Revised" License
6
stars
6
forks
source link
Fix macOS wheel build for `datajoint` (Issue #249) #406
Summary:
This PR addresses issue #249, which involves a wheel build failure for datajoint on macOS 12 when using mamba, and proposes three solutions for macOS users.
Also, this PR:
[x] Fix GitHub Actions (GHA) test Build env and run tests on macos-latest
[x] Refactor build_env_run_tests.yml workflow.
Context: On macOS, when attempting to install datajoint through a conda/mamba environment using an .yml file (see below), the following error occurs during the installation:
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [48 lines of output]
/Users/milagros/miniconda/envs/aeon_test/lib/python3.12/site-packages/setuptools/__init__.py:88: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!
********************************************************************************
Requirements should be satisfied by a PEP 517 installer.
If you are using pip, you can try `pip install --use-pep517`.
********************************************************************************
...
OSError: [Errno 66] Directory not empty: '/private/var/folders/h8/_lx50k3d6qx586662kr066h40000gn/T/pip-install-ovdbidfz/datajoint_530cc0190d4342fc91f8abda5f5fdf76/.eggs/watchdog-5.0.2-py3.12-macosx-11.0-arm64.egg/watchdog-5.0.2.dist-info' -> '/private/var/folders/h8/_lx50k3d6qx586662kr066h40000gn/T/pip-install-ovdbidfz/datajoint_530cc0190d4342fc91f8abda5f5fdf76/.eggs/watchdog-5.0.2-py3.12-macosx-11.0-arm64.egg/EGG-INFO'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed cleaning build dir for datajoint
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (datajoint)
failed
CondaEnvException: Pip failed
setuptools no longer recommends using setup.py for package installation. Specifically, setuptools is now raising warnings that setup.py install is deprecated in favor of using modern standards-based tools such as PEP 517 to build packages.
This deprecation is affecting how datajoint package is resolved during the build process for Apple Silicon (M1/M2) Macs. The python packages listed in the setup.py of datajoint dependency are not properly installed.
Solution: As python evolves to modernize package building and installation, the transition from setup.py to pyproject.toml (following PEP 517 and PEP 518) will help standardize the process. To stay aligned with these best practices, datajoint-python can consider updating its dependency management to use pyproject.toml for smoother builds and installations for mac users.
Temporal solutions are:
Build datajoint wheel using pip manually in the shell to bypass the issue:
pip install datajoint
Downgrading setuptools to version 58.2.0 would remove the warning but may create conflicts with other dependencies in the environment [1, 2]
Passing the --use-pep517 flag to pip to solve the wheel-building issue on macOS [3, 4]
Proposed solution in this PR for macOS installation process:
Create a env_macos.yml file for conda/mamba, excluding the pip installation of datajoint and datajoint-utilities.
After the conda environment is created, manually install both packages using pip from the shell within the conda/mamba environment, while passing the required flag. This is also indicated in the env_macos.yml for the macOS user reference.
Tests:
Locally using conda, and using GHA with mamba.
Summary: This PR addresses issue #249, which involves a wheel build failure for
datajoint
on macOS 12 when using mamba, and proposes three solutions for macOS users.Also, this PR:
Build env and run tests on macos-latest
build_env_run_tests.yml
workflow.Context: On macOS, when attempting to install
datajoint
through a conda/mamba environment using an.yml
file (see below), the following error occurs during the installation:setuptools
no longer recommends usingsetup.py
for package installation. Specifically,setuptools
is now raising warnings thatsetup.py
install is deprecated in favor of using modern standards-based tools such as PEP 517 to build packages.This deprecation is affecting how
datajoint
package is resolved during the build process for Apple Silicon (M1/M2) Macs. The python packages listed in thesetup.py
ofdatajoint
dependency are not properly installed.Solution: As python evolves to modernize package building and installation, the transition from
setup.py
topyproject.toml
(following PEP 517 and PEP 518) will help standardize the process. To stay aligned with these best practices,datajoint-python
can consider updating its dependency management to usepyproject.toml
for smoother builds and installations for mac users.Temporal solutions are:
datajoint
wheel using pip manually in the shell to bypass the issue:pip install datajoint
setuptools
to version 58.2.0 would remove the warning but may create conflicts with other dependencies in the environment [1, 2] Passing the--use-pep517
flag topip
to solve the wheel-building issue on macOS [3, 4]Proposed solution in this PR for macOS installation process:
env_macos.yml
file for conda/mamba, excluding the pip installation ofdatajoint
anddatajoint-utilities
.pip
from the shell within the conda/mamba environment, while passing the required flag. This is also indicated in theenv_macos.yml
for the macOS user reference.Tests: Locally using conda, and using GHA with mamba.
Environment details: macOS: 12, 14.5 python: 3.11, 3.12.6 conda/mamba: installation attempted via
conda/mamba env create -f env.yml
Main packages: