search

JohannesBuchner / UltraNest

Fit and compare complex models reliably and rapidly. Advanced nested sampling.
https://johannesbuchner.github.io/UltraNest/
Other
142 stars 30 forks source link

Ultranest 3.6.1 does not install in new .venv environment #102

Closed PieterVuylsteke closed 1 year ago

PieterVuylsteke commented 1 year ago

Description

I tried to install ultranest 3.6.1 in a new venv environment, it did not end succesfully 

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
              # only consider points in the same cluster
              if clusterids[j] == clusterids[i]:
                  pair_dist = 0.0
                  for k in range(ndim):
                      pair_dist += (pts[i,k] - pts[j,k])**2
                  total_dist += pair_dist**0.5
                  ^
  ------------------------------------------------------------

  ultranest\mlfriends.pyx:180:16: Cannot assign type 'npy_double complex' to 'float_t'

What I Did

Collecting scipy Obtaining dependency information for scipy from https://files.pythonhosted.org/packages/96/9b/10048be0c335327077af430c5a6637c0b9e7fe9121a8048836f1bb022a81/scipy-1.11.1-cp39-cp39-win_am Downloading scipy-1.11.1-cp39-cp39-win_amd64.whl.metadata (59 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 59.1/59.1 kB 1.6 MB/s eta 0:00:00 Collecting numpy<1.28.0,>=1.21.6 (from scipy) Obtaining dependency information for numpy<1.28.0,>=1.21.6 from https://files.pythonhosted.org/packages/86/c7/f92afdefa2bccdf0bc357321a931adafb1a999bc84f8877a6ed786a69ccc/numpy-1.25.1-cp39-cp39-win_amd64.whl.metadata Downloading numpy-1.25.1-cp39-cp39-win_amd64.whl.metadata (5.7 kB) Downloading scipy-1.11.1-cp39-cp39-win_amd64.whl (44.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 44.1/44.1 MB 1.2 MB/s eta 0:00:00 Downloading numpy-1.25.1-cp39-cp39-win_amd64.whl (15.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.1/15.1 MB 978.6 kB/s eta 0:00:00 Installing collected packages: numpy, scipy Successfully installed numpy-1.25.1 scipy-1.11.1 (.venv) PS C:\AstroPie_project\env_vak> & c:/AstroPie_project/env_vak/.venv/Scripts/pip install lmfit Collecting lmfit Obtaining dependency information for lmfit from https://files.pythonhosted.org/packages/16/87/d1b01fa4e4b69e0dea803592af88608c1c22eb4f92bfccf83ac66fa0792f/lmfit-1.2.2-py3-none-any.whl.metadata Downloading lmfit-1.2.2-py3-none-any.whl.metadata (9.9 kB) Collecting asteval>=0.9.28 (from lmfit) Obtaining dependency information for asteval>=0.9.28 from https://files.pythonhosted.org/packages/05/34/bdb51767967cb29302ee7dfe95662b057af7f23c62dd1967fc4b373656aa/asteval-0.9.31-py3-none-any.whl.metadata Downloading asteval-0.9.31-py3-none-any.whl.metadata (2.0 kB) Requirement already satisfied: numpy>=1.19 in c:\astropie_project\env_vak.venv\lib\site-packages (from lmfit) (1.25.1) Requirement already satisfied: scipy>=1.6 in c:\astropie_project\env_vak.venv\lib\site-packages (from lmfit) (1.11.1) Collecting uncertainties>=3.1.4 (from lmfit) Downloading uncertainties-3.1.7-py2.py3-none-any.whl (98 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.4/98.4 kB 1.4 MB/s eta 0:00:00 Collecting future (from uncertainties>=3.1.4->lmfit) Downloading future-0.18.3.tar.gz (840 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 840.9/840.9 kB 770.6 kB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Downloading lmfit-1.2.2-py3-none-any.whl (102 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 103.0/103.0 kB 850.6 kB/s eta 0:00:00 Downloading asteval-0.9.31-py3-none-any.whl (20 kB) Building wheels for collected packages: future Building wheel for future (pyproject.toml) ... done Created wheel for future: filename=future-0.18.3-py3-none-any.whl size=492053 sha256=fc66e06e748a38bf6039b1075a88eaa8059dbbe3448b75efb8dbadfcbd7ffb54 Stored in directory: c:\users\piete\appdata\local\pip\cache\wheels\bf\5d\6a\2e53874f7ec4e2bede522385439531fafec8fafe005b5c3d1b Successfully built future Installing collected packages: future, asteval, uncertainties, lmfit Successfully installed asteval-0.9.31 fu& c:/AstroPie_project/env_vak/.venv/Scripts/pip install pandasPS C:\AstroPie_project\env_vak> Collecting pandas Obtaining dependency information for pandas from https://files.pythonhosted.org/packages/9a/f2/0ad053856debbe90c83de1b4f05915f85fd2146f20faf9daa3b320d36df3/pandas-2.0.3-cp39-cp39-win_amd64.whl.metadata Downloading pandas-2.0.3-cp39-cp39-win_amd64.whl.metadata (18 kB) Requirement already satisfied: python-dateutil>=2.8.2 in c:\astropie_project\env_vak.venv\lib\site-packages (from pandas) (2.8.2) Downloading pytz-2023.3-py2.py3-none-any.whl (502 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 502.3/502.3 kB 955.5 kB/s eta 0:00:00 Collecting tzdata>=2022.1 (from pandas) Downloading tzdata-2023.3-py2.py3-none-any.whl (341 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 341.8/341.8 kB 848.1 kB/s eta 0:00:00 Requirement already satisfied: numpy>=1.20.3 in c:\astropie_project\env_vak.venv\lib\site-packages (from pandas) (1.25.1) Requirement already satisfied: six>=1.5 in c:\astropie_project\env_vak.venv\lib\site-packages (from python-dateutil>=2.8.2->pandas) (1.16.0) Downloading pandas-2.0.3-cp39-cp39-win_amd64.whl (10.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.8/10.8 MB 1.0 MB/s eta 0:00:00 Installing collected packages: pytz, tzdata, pandas Successfully installed pandas-2.0.3 pytz-2023.3 tzdata-2023.3 (.venv) PS C:\AstroPie_project\env_vak> & c:/AstroPie_project/env_vak/.venv/Scripts/pip install matplotlib Collecting matplotlib Obtaining dependency information for matplotlib from https://files.pythonhosted.org/packages/c9/46/6cbaf20f5bd0e7c1d204b45b853c2cd317b303fada90245f2825ecca47de/matplotlib-3.7.2-cp39-cp39-win_amd64.whl.metadata Downloading matplotlib-3.7.2-cp39-cp39-win_amd64.whl.metadata (5.8 kB) Collecting contourpy>=1.0.1 (from matplotlib) Obtaining dependency information for contourpy>=1.0.1 from https://files.pythonhosted.org/packages/a5/d6/80258c2759bd34abe267b5d3bc6300f7105aa70181b99d531283f7e7c79e/contourpy-1.1.0-cp39-cp39-win_amd64.whl.metadata Downloading contourpy-1.1.0-cp39-cp39-win_amd64.whl.metadata (5.7 kB) Collecting cycler>=0.10 (from matplotlib) Using cached cycler-0.11.0-py3-none-any.whl (6.4 kB) Collecting fonttools>=4.22.0 (from matplotlib) Obtaining dependency information for fonttools>=4.22.0 from https://files.pythonhosted.org/packages/50/4d/87db7f77a3ac0b4676dc34ae44437327546578ccb3f491196f2d816a8448/fonttools-4.41.0-cp39-cp39-win_amd64.whl.metadata Downloading fonttools-4.41.0-cp39-cp39-win_amd64.whl.metadata (152 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 152.5/152.5 kB 652.4 kB/s eta 0:00:00 Collecting kiwisolver>=1.0.1 (from matplotlib) Downloading kiwisolver-1.4.4-cp39-cp39-win_amd64.whl (55 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 55.4/55.4 kB 579.1 kB/s eta 0:00:00 Requirement already satisfied: numpy>=1.20 in c:\astropie_project\env_vak.venv\lib\site-packages (from matplotlib) (1.25.1) Requirement already satisfied: packaging>=20.0 in c:\astropie_project\env_vak.venv\lib\site-packages (from matplotlib) (23.1) Collecting pillow>=6.2.0 (from matplotlib) Obtaining dependency information for pillow>=6.2.0 from https://files.pythonhosted.org/packages/8f/b8/1bf1012eee3059d150194d1fab148f553f3df42cf412e4e6656c772afad9/Pillow-10.0.0-cp39-cp39-win_amd64.whl.metadata Downloading Pillow-10.0.0-cp39-cp39-win_amd64.whl.metadata (9.6 kB) Collecting pyparsing<3.1,>=2.3.1 (from matplotlib) Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.3/98.3 kB 624.8 kB/s eta 0:00:00 Requirement already satisfied: python-dateutil>=2.7 in c:\astropie_project\env_vak.venv\lib\site-packages (from matplotlib) (2.8.2) Collecting importlib-resources>=3.2.0 (from matplotlib) Obtaining dependency information for importlib-resources>=3.2.0 from https://files.pythonhosted.org/packages/29/d1/bed03eca30aa05aaf6e0873de091f9385c48705c4a607c2dfe3edbe543e8/importlib_resources-6.0.0-py3-none-any.whl.metadata Downloading importlib_resources-6.0.0-py3-none-any.whl.metadata (4.2 kB) Requirement already satisfied: zipp>=3.1.0 in c:\astropie_project\env_vak.venv\lib\site-packages (from importlib-resources>=3.2.0->matplotlib) (3.16.2) Requirement already satisfied: six>=1.5 in c:\astropie_project\env_vak.venv\lib\site-packagesDownloading matplotlib-3.7.2-cp39-cp39-win_amd64.whl (7.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.5/7.5 MB 981.1 kB/s eta 0:00:00 Downloading contourpy-1.1.0-cp39-cp39-win_amd64.whl (429 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 429.4/429.4 kB 1.4 MB/s eta 0:00:00 Downloading fonttools-4.41.0-cp39-cp39-win_amd64.whl (2.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 1.1 MB/s eta 0:00:00 Downloading importlib_resources-6.0.0-py3-none-any.whl (31 kB) Downloading Pillow-10.0.0-cp39-cp39-win_amd64.whl (2.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.5/2.5 MB 1.5 MB/s eta 0:00:00 Installing collected packages: pyparsing, pillow, kiwisolver, importlib-resources, fonttools, cycler, contourpy, matplotlib Successfully installed contourpy-1.1.0 cycler-0.11.0 fonttools-4.41.0 importlib-resources-6.0.0 kiwisolver-1.4.4 matplotlib-3.7.2 pillow-10.0.0 pyparsing-3.0.9 (.venv) PS C:\AstroPie_project\env_vak> & c:/AstroPie_project/env_vak/.venv/Scripts/pip install ultranest Collecting ultranest Using cached ultranest-3.6.1.tar.gz (21.5 MB) Installing build dependencies ... done Getting requirements to build wheel ... error error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> [38 lines of output]

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
              # only consider points in the same cluster
              if clusterids[j] == clusterids[i]:
                  pair_dist = 0.0
                  for k in range(ndim):
                      pair_dist += (pts[i,k] - pts[j,k])**2
                  total_dist += pair_dist**0.5
                  ^
  ------------------------------------------------------------

  ultranest\mlfriends.pyx:180:16: Cannot assign type 'npy_double complex' to 'float_t'    
  Compiling ultranest/mlfriends.pyx because it changed.
  Compiling ultranest/stepfuncs.pyx because it changed.
  [1/2] Cythonizing ultranest/mlfriends.pyx
  Traceback (most recent call last):
    File "c:\AstroPie_project\env_vak\.venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
      main()
    File "c:\AstroPie_project\env_vak\.venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "c:\AstroPie_project\env_vak\.venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
    File "C:\Users\piete\AppData\Local\Temp\pip-build-env-yq_n6z7s\overlay\Lib\site-packages\setuptools\build_meta.py", line 341, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "C:\Users\piete\AppData\Local\Temp\pip-build-env-yq_n6z7s\overlay\Lib\site-packages\setuptools\build_meta.py", line 323, in _get_build_requires
      self.run_setup()
    File "C:\Users\piete\AppData\Local\Temp\pip-build-env-yq_n6z7s\overlay\Lib\site-packages\setuptools\build_meta.py", line 487, in run_setup
      super(_BuildMetaLegacyBackend,
    File "C:\Users\piete\AppData\Local\Temp\pip-build-env-yq_n6z7s\overlay\Lib\site-packages\setuptools\build_meta.py", line 338, in run_setup
      exec(code, locals())
    File "<string>", line 58, in <module>
    File "C:\Users\piete\AppData\Local\Temp\pip-build-env-yq_n6z7s\overlay\Lib\site-packages\Cython\Build\Dependencies.py", line 1134, in cythonize
      cythonize_one(*args)
    File "C:\Users\piete\AppData\Local\Temp\pip-build-env-yq_n6z7s\overlay\Lib\site-packagges\Cython\Build\Dependencies.py", line 1301, in cythonize_one
      raise CompileError(None, pyx_file)
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output.

JohannesBuchner commented 1 year ago

weird: the sum of squares, pair_dist, seems to have become negative!? Then the sqrt (**0.5) makes it complex.

jaytmiller commented 1 year ago

I had the same issue. I was able to work around it by forking and then modifying pyproject.toml to pin cython==0.29.36 but I am curious if there's a better fix and would definitely prefer to be installing from PyPi or your git repo.

JohannesBuchner commented 1 year ago

Would total_dist += pair_dist**0.5).real compile? I wonder if that then breaks with older cython versions instead?

ahnitz commented 1 year ago

@JohannesBuchner For reference, I think the cython3 relevant change is what is noted at the end of this page. https://cython.readthedocs.io/en/latest/src/userguide/migrating_to_cy30.html#power-operator

JohannesBuchner commented 1 year ago

Thanks a lot @ahnitz .

So compute_mean_pair_distance should be annotated with @cython.cpow(True)

PieterVuylsteke commented 1 year ago

we changed it in this way : on line 12 :from cython.cimports.libc.math import sqrt

and then also on line 180 : total_dist += sqrt(pair_dist)

JohannesBuchner commented 1 year ago

That is probably better. Can you please make a pull request, @PieterVuylsteke ?

ahnitz commented 1 year ago

@JohannesBuchner @PieterVuylsteke That should work. I just tested an alternative which may be useful for future reference. The tope-level compiler directive works works as well and seems to be backwards compatible before the cpow directive was added. Switching to sqrt would seem to be just as good if not better though.

ahnitz commented 1 year ago

@JohannesBuchner If possible, once some patch is accepted for this issue, could I request a minor point release or UltraNest, just so that people installing from pypi don't immediately get hit by an install issue.

JohannesBuchner commented 1 year ago

Sure. First I need to wait for the CI to OK it https://app.circleci.com/pipelines/github/JohannesBuchner/UltraNest/371/workflows/2f4d13cc-8da4-4c78-a313-71ed09f13eb1/jobs/352

PieterVuylsteke commented 1 year ago

got

That is probably better. Can you please make a pull request, @PieterVuylsteke ?

seems its done already. no problemo.

JohannesBuchner commented 1 year ago

UltraNest version 3.6.2 is on pypi, please test it and let me know.

Thanks @PieterVuylsteke for reporting the issue and providing a solution.

PieterVuylsteke commented 1 year ago

3.6.2 works like a charm

jpl-jengelke commented 1 year ago

This was due to the major Cython upgrade from 0.x to 3.x and because Cython is specified in the requires metadata of pyproject.toml it cannot be overridden (AFAIK).

Posting this link to record it in the Cython issue logs: https://github.com/cython/cython/issues/5540

A recipe for installing older Cython versions is provided in #105.