BUG - unable to install ``celer`` in an empty python virtual environment

[Badr-MOUFAD]

minimal steps to reproduce,

1. cd to a new directory
2. create a python environment python -m venv venv
3. install celer pip install -U celer

Error logs

(click to expend)

import numpy.distutils.command.sdist Traceback (most recent call last): File "C:\Users\HP\AppData\Local\Temp\easy_install-unrz350b\numpy-1.23.0rc3\setup.py", line 251, in generate_cython ModuleNotFoundError: No module named 'Cython' The above exception was the direct cause of the following exception: Traceback (most recent call last): File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 154, in save_modules yield saved File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 195, in setup_context yield File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 250, in run_setup _execfile(setup_script, ns) File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 45, in _execfile exec(code, globals, locals) File "C:\Users\HP\AppData\Local\Temp\easy_install-unrz350b\numpy-1.23.0rc3\setup.py", line 493, in File "C:\Users\HP\AppData\Local\Temp\easy_install-unrz350b\numpy-1.23.0rc3\setup.py", line 475, in setup_package File "C:\Users\HP\AppData\Local\Temp\easy_install-unrz350b\numpy-1.23.0rc3\setup.py", line 258, in generate_cython OSError: Cython needs to be installed in Python as a module During handling of the above exception, another exception occurred: Traceback (most recent call last): File "", line 2, in File "", line 34, in File "C:\Users\HP\AppData\Local\Temp\pip-req-build-87kkcnrt\setup.py", line 4, in dist.Distribution().fetch_build_eggs(['numpy>=1.12']) File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\dist.py", line 716, in fetch_build_eggs resolved_dists = pkg_resources.working_set.resolve( File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\pkg_resources\__init__.py", line 780, in resolve dist = best[req.key] = env.best_match( File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\pkg_resources\__init__.py", line 1065, in best_match return self.obtain(req, installer) File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\pkg_resources\__init__.py", line 1077, in obtain return installer(requirement) File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\dist.py", line 786, in fetch_build_egg return cmd.easy_install(req) File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\command\easy_install.py", line 679, in easy_install return self.install_item(spec, dist.location, tmpdir, deps) File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\command\easy_install.py", line 705, in install_item dists = self.install_eggs(spec, download, tmpdir) File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\command\easy_install.py", line 890, in install_eggs return self.build_and_install(setup_script, setup_base) File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\command\easy_install.py", line 1158, in build_and_install self.run_setup(setup_script, setup_base, args) File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\command\easy_install.py", line 1144, in run_setup run_setup(setup_script, args) File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 253, in run_setup raise File "C:\Users\HP\AppData\Local\Programs\Python\Python38-32\lib\contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 195, in setup_context yield File "C:\Users\HP\AppData\Local\Programs\Python\Python38-32\lib\contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 166, in save_modules saved_exc.resume() File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 141, in resume six.reraise(type, exc, self._tb) File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\_vendor\six.py", line 685, in reraise raise value.with_traceback(tb) File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 154, in save_modules yield saved File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 195, in setup_context yield File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 250, in run_setup _execfile(setup_script, ns) File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 45, in _execfile exec(code, globals, locals) File "C:\Users\HP\AppData\Local\Temp\easy_install-unrz350b\numpy-1.23.0rc3\setup.py", line 493, in File "C:\Users\HP\AppData\Local\Temp\easy_install-unrz350b\numpy-1.23.0rc3\setup.py", line 475, in setup_package File "C:\Users\HP\AppData\Local\Temp\easy_install-unrz350b\numpy-1.23.0rc3\setup.py", line 258, in generate_cython OSError: Cython needs to be installed in Python as a module [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output.

Additional comments

It seems that the error is related to fetching NumPy builds

https://github.com/mathurinm/celer/blob/bd11a44471ff5b688eea30bb43030ec547dd3a4d/setup.py#L4

Also, a portion of the error, namely

OSError: Cython needs to be installed in Python as a module
      [end of output]

is related to https://github.com/numpy/numpy/blob/705244444526804860e9f8d52459e2ca5c255366/setup.py#L257-L258

[Badr-MOUFAD]

@mathurinm could you pls verify whether this bug is OS related?

[mathurinm]

I use conda envs not virtual envs, can you give me steps to reproduce ?

Is it related to the wheel buidling we released in 0.7 ? Does pip install celer --no-binary work? And with version 6.1 ?

If it fails in an empty env it will fail in CI's of benchmarks were celer is a requirement, we can check that too

[Badr-MOUFAD]

I use conda envs not virtual envs, can you give me steps to reproduce ?

The bug doesn't occur in conda environment.

Is it related to the wheel buidling we released in 0.7 ?

No, we also get the same bug for previous versions.

[Badr-MOUFAD]

This was solved after releasing wheels (https://github.com/mathurinm/celer/pull/274) for Windows, macOS, and Linux.