maindlt commented 5 months ago

To make DYNAMITE install on Python 3.12, the following changes are applied:

numpy requirements changed from numpy<1.25,>=1.21 to numpy>=1.21. The complicated reason: Older numpy versions somehow use a deprecated attribute ImpImporter of the module pkgutil and that attribute is removed in Python 3.12. The previous limitation numpy<1.25 had been necessary because of numba that was not compatible with newer numpy versions (numba is an implicit requirement of the astropy package which DYNAMITE requires). Current numba versions don't have this limitation (tested with Python 3.8...3.12).
Directly executing setup.py has been deprecated for a while. The alternative is to use pip which implicitly reads the information in setup.py. The new procedure to install DYNAMITE (along with the legacy_fortran executables) is via the command python -m pip install . or python -m pip install . --user. This has been tested with Python 3.8...3.12, is now in the docs, but needs some more testing and the docs need some polishing (the old text is still there, too).
setup.py uses setuptools. This is ok, but the environment variable SETUPTOOLS_USE_DISTUTILS must not be set (or set to the empty string), otherwise setuptools use distutils which is deprecated and causes the pip installation to crash with Python 3.12. An according note has been added to the docs.

Given the significant changes in how to install DYNAMITE, let's discuss in (one of) the next hack session(s) and test thoroughly...

Closes #381.

maindlt commented 5 months ago

Feedback by Julia: DYNAMITE installs with Python 3.12 in a conda environment on the cluster if

in requirements.txt, numpy<1.25,>=1.21 is replaced by numpy>=1.21
the environment variable SETUPTOOLS_USE_DISTUTILS is not set
installation is done via python -m pip install .

prashjet commented 5 months ago

Installed in a fresh conda env with python 3.12, and test_notebooks passed successfully

boeckera commented 5 months ago

Tested with python 3.11.5 and notebooks passed, but failed to install dynamite with a fresh conda env python 3.8. It gives me an error that the cvxopt library cannot be build.

Processing /Users/alinaboecker/Documents/dynamite Preparing metadata (setup.py) ... done Collecting astropy>=5.0.4 (from dynamite==4.2.0) Downloading astropy-5.2.2-cp38-cp38-macosx_11_0_arm64.whl.metadata (8.2 kB) Collecting cmasher>=1.6.0 (from dynamite==4.2.0) Downloading cmasher-1.6.3-py3-none-any.whl.metadata (8.2 kB) Collecting cvxopt>=1.2.6 (from dynamite==4.2.0) Downloading cvxopt-1.3.2.tar.gz (4.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.1/4.1 MB 9.2 MB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing metadata (pyproject.toml) ... done Collecting h5py>=3.1.0 (from dynamite==4.2.0) Downloading h5py-3.11.0-cp38-cp38-macosx_11_0_arm64.whl.metadata (2.5 kB) Collecting lmfit (from dynamite==4.2.0) Downloading lmfit-1.3.1-py3-none-any.whl.metadata (13 kB) Collecting matplotlib>=3.3.4 (from dynamite==4.2.0) Downloading matplotlib-3.7.5-cp38-cp38-macosx_11_0_arm64.whl.metadata (5.7 kB) Collecting numpy>=1.21 (from dynamite==4.2.0) Downloading numpy-1.24.4-cp38-cp38-macosx_11_0_arm64.whl.metadata (5.6 kB) Collecting pafit>=2.0.8 (from dynamite==4.2.0) Downloading pafit-2.0.8.tar.gz (7.3 kB) Preparing metadata (setup.py) ... done Collecting pathos>=0.2.7 (from dynamite==4.2.0) Downloading pathos-0.3.2-py3-none-any.whl.metadata (11 kB) Collecting plotbin>=3.1.3 (from dynamite==4.2.0) Downloading plotbin-3.1.5.tar.gz (10 kB) Preparing metadata (setup.py) ... done Collecting PyYAML>=5.4.1 (from dynamite==4.2.0) Downloading PyYAML-6.0.1.tar.gz (125 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.2/125.2 kB 8.6 MB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Collecting scipy>=1.10 (from dynamite==4.2.0) Downloading scipy-1.10.1-cp38-cp38-macosx_12_0_arm64.whl.metadata (53 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.9/53.9 kB 4.7 MB/s eta 0:00:00 Collecting six (from dynamite==4.2.0) Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) Collecting sparse>=0.14.0 (from dynamite==4.2.0) Downloading sparse-0.15.1-py2.py3-none-any.whl.metadata (4.5 kB) Collecting vorbin>=3.1.4 (from dynamite==4.2.0) Downloading vorbin-3.1.5.tar.gz (91 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 91.5/91.5 kB 7.8 MB/s eta 0:00:00 Preparing metadata (setup.py) ... done Collecting pyerfa>=2.0 (from astropy>=5.0.4->dynamite==4.2.0) Downloading pyerfa-2.0.0.3-cp38-cp38-macosx_11_0_arm64.whl.metadata (5.6 kB) Collecting packaging>=19.0 (from astropy>=5.0.4->dynamite==4.2.0) Using cached packaging-24.0-py3-none-any.whl.metadata (3.2 kB) Collecting colorspacious>=1.1.0 (from cmasher>=1.6.0->dynamite==4.2.0) Downloading colorspacious-1.1.2-py2.py3-none-any.whl.metadata (3.6 kB) Collecting e13tools>=0.9.4 (from cmasher>=1.6.0->dynamite==4.2.0) Downloading e13tools-0.9.6-py3-none-any.whl.metadata (2.9 kB) Collecting contourpy>=1.0.1 (from matplotlib>=3.3.4->dynamite==4.2.0) Downloading contourpy-1.1.1-cp38-cp38-macosx_11_0_arm64.whl.metadata (5.9 kB) Collecting cycler>=0.10 (from matplotlib>=3.3.4->dynamite==4.2.0) Downloading cycler-0.12.1-py3-none-any.whl.metadata (3.8 kB) Collecting fonttools>=4.22.0 (from matplotlib>=3.3.4->dynamite==4.2.0) Downloading fonttools-4.51.0-cp38-cp38-macosx_10_9_universal2.whl.metadata (159 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 159.5/159.5 kB 11.5 MB/s eta 0:00:00 Collecting kiwisolver>=1.0.1 (from matplotlib>=3.3.4->dynamite==4.2.0) Downloading kiwisolver-1.4.5-cp38-cp38-macosx_11_0_arm64.whl.metadata (6.4 kB) Collecting pillow>=6.2.0 (from matplotlib>=3.3.4->dynamite==4.2.0) Downloading pillow-10.3.0-cp38-cp38-macosx_11_0_arm64.whl.metadata (9.2 kB) Collecting pyparsing>=2.3.1 (from matplotlib>=3.3.4->dynamite==4.2.0) Downloading pyparsing-3.1.2-py3-none-any.whl.metadata (5.1 kB) Collecting python-dateutil>=2.7 (from matplotlib>=3.3.4->dynamite==4.2.0) Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) Collecting importlib-resources>=3.2.0 (from matplotlib>=3.3.4->dynamite==4.2.0) Downloading importlib_resources-6.4.0-py3-none-any.whl.metadata (3.9 kB) Collecting ppft>=1.7.6.8 (from pathos>=0.2.7->dynamite==4.2.0) Downloading ppft-1.7.6.8-py3-none-any.whl.metadata (12 kB) Collecting dill>=0.3.8 (from pathos>=0.2.7->dynamite==4.2.0) Downloading dill-0.3.8-py3-none-any.whl.metadata (10 kB) Collecting pox>=0.3.4 (from pathos>=0.2.7->dynamite==4.2.0) Downloading pox-0.3.4-py3-none-any.whl.metadata (8.0 kB) Collecting multiprocess>=0.70.16 (from pathos>=0.2.7->dynamite==4.2.0) Downloading multiprocess-0.70.16-py38-none-any.whl.metadata (7.1 kB) Collecting numba>=0.49 (from sparse>=0.14.0->dynamite==4.2.0) Downloading numba-0.58.1-cp38-cp38-macosx_11_0_arm64.whl.metadata (2.7 kB) Collecting asteval>=0.9.28 (from lmfit->dynamite==4.2.0) Downloading asteval-0.9.32-py3-none-any.whl.metadata (2.0 kB) Collecting uncertainties>=3.1.4 (from lmfit->dynamite==4.2.0) Downloading uncertainties-3.1.7-py2.py3-none-any.whl.metadata (14 kB) Collecting zipp>=3.1.0 (from importlib-resources>=3.2.0->matplotlib>=3.3.4->dynamite==4.2.0) Downloading zipp-3.18.1-py3-none-any.whl.metadata (3.5 kB) Collecting llvmlite<0.42,>=0.41.0dev0 (from numba>=0.49->sparse>=0.14.0->dynamite==4.2.0) Downloading llvmlite-0.41.1-cp38-cp38-macosx_11_0_arm64.whl.metadata (4.8 kB) Collecting importlib-metadata (from numba>=0.49->sparse>=0.14.0->dynamite==4.2.0) Downloading importlib_metadata-7.1.0-py3-none-any.whl.metadata (4.7 kB) Collecting future (from uncertainties>=3.1.4->lmfit->dynamite==4.2.0) Downloading future-1.0.0-py3-none-any.whl.metadata (4.0 kB) Downloading astropy-5.2.2-cp38-cp38-macosx_11_0_arm64.whl (6.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.8/6.8 MB 5.6 MB/s eta 0:00:00 Downloading cmasher-1.6.3-py3-none-any.whl (367 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 367.1/367.1 kB 16.6 MB/s eta 0:00:00 Downloading h5py-3.11.0-cp38-cp38-macosx_11_0_arm64.whl (2.9 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.9/2.9 MB 14.2 MB/s eta 0:00:00 Downloading matplotlib-3.7.5-cp38-cp38-macosx_11_0_arm64.whl (7.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.3/7.3 MB 20.1 MB/s eta 0:00:00 Downloading numpy-1.24.4-cp38-cp38-macosx_11_0_arm64.whl (13.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.8/13.8 MB 20.2 MB/s eta 0:00:00 Downloading pathos-0.3.2-py3-none-any.whl (82 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 82.1/82.1 kB 6.3 MB/s eta 0:00:00 Downloading scipy-1.10.1-cp38-cp38-macosx_12_0_arm64.whl (28.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 28.8/28.8 MB 18.1 MB/s eta 0:00:00 Downloading sparse-0.15.1-py2.py3-none-any.whl (116 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 116.3/116.3 kB 9.3 MB/s eta 0:00:00 Downloading lmfit-1.3.1-py3-none-any.whl (105 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 105.9/105.9 kB 11.1 MB/s eta 0:00:00 Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) Downloading asteval-0.9.32-py3-none-any.whl (20 kB) Downloading colorspacious-1.1.2-py2.py3-none-any.whl (37 kB) Downloading contourpy-1.1.1-cp38-cp38-macosx_11_0_arm64.whl (232 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 232.4/232.4 kB 13.7 MB/s eta 0:00:00 Downloading cycler-0.12.1-py3-none-any.whl (8.3 kB) Downloading dill-0.3.8-py3-none-any.whl (116 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 116.3/116.3 kB 12.5 MB/s eta 0:00:00 Downloading e13tools-0.9.6-py3-none-any.whl (40 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.9/40.9 kB 2.8 MB/s eta 0:00:00 Downloading fonttools-4.51.0-cp38-cp38-macosx_10_9_universal2.whl (2.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.8/2.8 MB 16.4 MB/s eta 0:00:00 Downloading importlib_resources-6.4.0-py3-none-any.whl (38 kB) Downloading kiwisolver-1.4.5-cp38-cp38-macosx_11_0_arm64.whl (66 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.0/66.0 kB 6.6 MB/s eta 0:00:00 Downloading multiprocess-0.70.16-py38-none-any.whl (132 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.6/132.6 kB 9.6 MB/s eta 0:00:00 Downloading numba-0.58.1-cp38-cp38-macosx_11_0_arm64.whl (2.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.6/2.6 MB 18.3 MB/s eta 0:00:00 Using cached packaging-24.0-py3-none-any.whl (53 kB) Downloading pillow-10.3.0-cp38-cp38-macosx_11_0_arm64.whl (3.4 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 19.0 MB/s eta 0:00:00 Downloading pox-0.3.4-py3-none-any.whl (29 kB) Downloading ppft-1.7.6.8-py3-none-any.whl (56 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.8/56.8 kB 5.0 MB/s eta 0:00:00 Downloading pyerfa-2.0.0.3-cp38-cp38-macosx_11_0_arm64.whl (333 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 333.2/333.2 kB 21.4 MB/s eta 0:00:00 Downloading pyparsing-3.1.2-py3-none-any.whl (103 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 103.2/103.2 kB 8.6 MB/s eta 0:00:00 Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 229.9/229.9 kB 12.2 MB/s eta 0:00:00 Downloading uncertainties-3.1.7-py2.py3-none-any.whl (98 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.4/98.4 kB 7.8 MB/s eta 0:00:00 Downloading llvmlite-0.41.1-cp38-cp38-macosx_11_0_arm64.whl (28.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 28.8/28.8 MB 18.8 MB/s eta 0:00:00 Downloading zipp-3.18.1-py3-none-any.whl (8.2 kB) Downloading future-1.0.0-py3-none-any.whl (491 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 491.3/491.3 kB 16.2 MB/s eta 0:00:00 Downloading importlib_metadata-7.1.0-py3-none-any.whl (24 kB) Building wheels for collected packages: dynamite, cvxopt, pafit, plotbin, PyYAML, vorbin Building wheel for dynamite (setup.py) ... done Created wheel for dynamite: filename=dynamite-4.2.0-py3-none-any.whl size=3526254 sha256=c52565a08c2093a0d2d2c41d8f96ccacd7f30ab127f8e5ccef1ce05d355240ea Stored in directory: /private/var/folders/p1/qjv7_q354yzftsnt8bl2n2m40000gq/T/pip-ephem-wheel-cache-wqee9boa/wheels/0d/34/a8/7efd6be033392877bf8c029642bc35de27f121b2750a867fd3 Building wheel for cvxopt (pyproject.toml) ... error error: subprocess-exited-with-error

× Building wheel for cvxopt (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [38 lines of output] running bdist_wheel running build running build_py creating build creating build/lib.macosx-11.1-arm64-cpython-38 creating build/lib.macosx-11.1-arm64-cpython-38/cvxopt copying src/python/misc.py -> build/lib.macosx-11.1-arm64-cpython-38/cvxopt copying src/python/_version.py -> build/lib.macosx-11.1-arm64-cpython-38/cvxopt copying src/python/msk.py -> build/lib.macosx-11.1-arm64-cpython-38/cvxopt copying src/python/init.py -> build/lib.macosx-11.1-arm64-cpython-38/cvxopt copying src/python/solvers.py -> build/lib.macosx-11.1-arm64-cpython-38/cvxopt copying src/python/cvxprog.py -> build/lib.macosx-11.1-arm64-cpython-38/cvxopt copying src/python/modeling.py -> build/lib.macosx-11.1-arm64-cpython-38/cvxopt copying src/python/info.py -> build/lib.macosx-11.1-arm64-cpython-38/cvxopt copying src/python/coneprog.py -> build/lib.macosx-11.1-arm64-cpython-38/cvxopt copying src/python/printing.py -> build/lib.macosx-11.1-arm64-cpython-38/cvxopt running build_ext building 'base' extension creating build/temp.macosx-11.1-arm64-cpython-38 creating build/temp.macosx-11.1-arm64-cpython-38/src creating build/temp.macosx-11.1-arm64-cpython-38/src/C gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/alinaboecker/anaconda3/envs/dynamite_new_install/include -arch arm64 -I/Users/alinaboecker/anaconda3/envs/dynamite_new_install/include -arch arm64 -I/Users/alinaboecker/anaconda3/envs/dynamite_new_install/include/python3.8 -c src/C/base.c -o build/temp.macosx-11.1-arm64-cpython-38/src/C/base.o gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/alinaboecker/anaconda3/envs/dynamite_new_install/include -arch arm64 -I/Users/alinaboecker/anaconda3/envs/dynamite_new_install/include -arch arm64 -I/Users/alinaboecker/anaconda3/envs/dynamite_new_install/include/python3.8 -c src/C/dense.c -o build/temp.macosx-11.1-arm64-cpython-38/src/C/dense.o gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/alinaboecker/anaconda3/envs/dynamite_new_install/include -arch arm64 -I/Users/alinaboecker/anaconda3/envs/dynamite_new_install/include -arch arm64 -I/Users/alinaboecker/anaconda3/envs/dynamite_new_install/include/python3.8 -c src/C/sparse.c -o build/temp.macosx-11.1-arm64-cpython-38/src/C/sparse.o gcc -bundle -undefined dynamic_lookup -L/Users/alinaboecker/anaconda3/envs/dynamite_new_install/lib -arch arm64 -L/Users/alinaboecker/anaconda3/envs/dynamite_new_install/lib -arch arm64 build/temp.macosx-11.1-arm64-cpython-38/src/C/base.o build/temp.macosx-11.1-arm64-cpython-38/src/C/dense.o build/temp.macosx-11.1-arm64-cpython-38/src/C/sparse.o -L/usr/lib -lm -llapack -lblas -o build/lib.macosx-11.1-arm64-cpython-38/cvxopt/base.cpython-38-darwin.so building 'blas' extension gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/alinaboecker/anaconda3/envs/dynamite_new_install/include -arch arm64 -I/Users/alinaboecker/anaconda3/envs/dynamite_new_install/include -arch arm64 -I/Users/alinaboecker/anaconda3/envs/dynamite_new_install/include/python3.8 -c src/C/blas.c -o build/temp.macosx-11.1-arm64-cpython-38/src/C/blas.o gcc -bundle -undefined dynamic_lookup -L/Users/alinaboecker/anaconda3/envs/dynamite_new_install/lib -arch arm64 -L/Users/alinaboecker/anaconda3/envs/dynamite_new_install/lib -arch arm64 build/temp.macosx-11.1-arm64-cpython-38/src/C/blas.o -L/usr/lib -lblas -o build/lib.macosx-11.1-arm64-cpython-38/cvxopt/blas.cpython-38-darwin.so building 'lapack' extension gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/alinaboecker/anaconda3/envs/dynamite_new_install/include -arch arm64 -I/Users/alinaboecker/anaconda3/envs/dynamite_new_install/include -arch arm64 -I/Users/alinaboecker/anaconda3/envs/dynamite_new_install/include/python3.8 -c src/C/lapack.c -o build/temp.macosx-11.1-arm64-cpython-38/src/C/lapack.o gcc -bundle -undefined dynamic_lookup -L/Users/alinaboecker/anaconda3/envs/dynamite_new_install/lib -arch arm64 -L/Users/alinaboecker/anaconda3/envs/dynamite_new_install/lib -arch arm64 build/temp.macosx-11.1-arm64-cpython-38/src/C/lapack.o -L/usr/lib -llapack -lblas -o build/lib.macosx-11.1-arm64-cpython-38/cvxopt/lapack.cpython-38-darwin.so building 'umfpack' extension gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/alinaboecker/anaconda3/envs/dynamite_new_install/include -arch arm64 -I/Users/alinaboecker/anaconda3/envs/dynamite_new_install/include -arch arm64 -I/usr/local/include -I/Users/alinaboecker/anaconda3/envs/dynamite_new_install/include/python3.8 -c src/C/umfpack.c -o build/temp.macosx-11.1-arm64-cpython-38/src/C/umfpack.o src/C/umfpack.c:23:10: fatal error: 'umfpack.h' file not found

include "umfpack.h"

           ^~~~~~~~~~~
  1 error generated.
  error: command '/usr/bin/gcc' failed with exit code 1
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for cvxopt Building wheel for pafit (setup.py) ... done Created wheel for pafit: filename=pafit-2.0.8-py3-none-any.whl size=10684 sha256=72a6d94b86f81148b1ac7ad753789fb167fd47571a128c5148f1cbda17b197c9 Stored in directory: /Users/alinaboecker/Library/Caches/pip/wheels/46/33/f1/5faff7cdb2c9ed10d241c62ff6b83b1076ad53a9f20e4e2e8b Building wheel for plotbin (setup.py) ... done Created wheel for plotbin: filename=plotbin-3.1.5-py3-none-any.whl size=12895 sha256=54db994eff36a7f465c1ef34fbd7691eb48a7edcf2e6a1fa8bb222ab05aff2d2 Stored in directory: /Users/alinaboecker/Library/Caches/pip/wheels/57/47/55/59cc0721b632d64edfe79d7b4cb700fdd1b78f130b6df3404c Building wheel for PyYAML (pyproject.toml) ... done Created wheel for PyYAML: filename=PyYAML-6.0.1-cp38-cp38-macosx_11_0_arm64.whl size=45365 sha256=41515dcefdf374347fa998b6cf66cd4d7aef246e4930f0d5907ac27583273579 Stored in directory: /Users/alinaboecker/Library/Caches/pip/wheels/77/54/77/68b3079bd1d88cb070513c3935d9f7e32c70ad69368375308d Building wheel for vorbin (setup.py) ... done Created wheel for vorbin: filename=vorbin-3.1.5-py3-none-any.whl size=91491 sha256=c496bc026e6113fbcb27e93c1769485d50d5c00e965816267cd37081267907f5 Stored in directory: /Users/alinaboecker/Library/Caches/pip/wheels/4d/cf/9e/77f8b88b918046ab89e63629fff9f40626b16d6d12aedcef26 Successfully built dynamite pafit plotbin PyYAML vorbin Failed to build cvxopt ERROR: Could not build wheels for cvxopt, which is required to install pyproject.toml-based projects

boeckera commented 5 months ago

it seems to be connected to this https://github.com/cvxopt/cvxopt/issues/78 but no clue how to fix it

afeldmei commented 5 months ago

Installation with fresh conda env and python 3.10.14, test_notebooks.sh passed

maindlt commented 5 months ago

it seems to be connected to this cvxopt/cvxopt#78 but no clue how to fix it

Hi @boeckera,

many thanks for testing! The thing is that pip does find a binary wheel distribution of cvxopt for Python 3.11 on your platform (the.whl file), but does not find one for Python 3.8. A list of available platforms is here. macOS ARM64 is only supported for Python 3.9 or later. So it downloads the source files (.tar.gz) and tries to compile, which fails. On my macOS x86-64 machine a cvxopt wheel is available for Python 3.8…3.12, but when I force it to recompile (python -m pip install . --no-binary cvxopt --force-reinstall) I get the same error you are getting. From what I see in your linked issue from the cvxopt git repo, there seem to be a number of steps necessary to get it compiled from source. Frankly, I do not think that we should go into supporting this, for the following reasons:

Supporting the building of 3rd party modules uses a lot of resources on our side (see e.g., GALAHAD…)
cvxopt is only one of two options for the Python NNLS solver (the other one being scipy.optimize)
I assume that there is a wheel available for cvxopt for any recent system (as it is the case for Python 3.11 on your machine, for Python 3.8…3.12 on my older machine, and 3.10, 3.12 on the cluster)

The question is how we deal with the fact that users may not be able to install DYNAMITE because of cvxopt:

Just mention a possible issue with building cvxopt in the docs, recommending to use a more recent version of Python and linking to the issue on the cvxopt git.
Make cvxopt an optional requirement in setup.py (extras_require). However, this would mean that the user has to explicitly include cvxopt when installing (via a parameter to pip) and the code would need to made aware that the package might be there or not (in the config reader and the weight solver).
Require the user to install cvxopt separately (again, the DYNAMITE code would need to be adapted).

I would opt for the first solution, expecting that most users will be able to install DYNAMITE ok (I cannot remember that anyone had problems with cvxopt so far), what do you think?

Cheers, Thomas

maindlt commented 3 months ago

Per core team meeting: Go with option 2: Make cvxopt an optional requirement, mention this in the docs

maindlt commented 3 months ago

Hi @boeckera, here come some ideas for testing (as discussed today):

In a clean environment, install DYNAMITE viapython -m pip install . (executed in the "main" dynamite/ directory). This does not install cvxopt because it is now optional.
- Try to execute test_nnls.py with type: 'NNLS' and nnls_solver: 'cvxopt' in its config file user_test_config_ml.yaml. DYNAMITE should crash and the error message should give a hint on what to do (different solver or install cvxopt).
- Change the weight solver in the config file to nnls_solver: 'scipy' and run test_nnls.py again. It should finish without an error.
- Now, install DYNAMITE viapython -m pip install .[cvxopt] (executed in the "main" dynamite/ directory). This install includes the optional requirement cvxopt and will only succeed if cvxopt can be installed on your system "out of the box".
- Run test_nnls.py with both nnls_solver: 'cvxopt' and nnls_solver: 'scipy' in the config file. Both should complete without an error.
- Do this for Python 3.12 and at least one earlier version >=3.8.
Please also have a quick look at the docs, ideally via the "Files changed" tab above to see if the changes make sense ;-)

Many thanks for taking the time for testing :-)

maindlt commented 1 week ago

Will merge this PR given the positive comments above by multiple users...

dynamics-of-stellar-systems / dynamite

New installation procedure #383

include "umfpack.h"