ModuleNotFoundError: No module named 'setuptools'

[oroppas]

Under venv, pip fails despite the fact that setuptools is installed

tree -L 1 venv/lib/python3.8/site-packages
venv/lib/python3.8/site-packages
├── easy_install.py
├── pip
├── pip-20.1.1.dist-info
├── pkg_resources
├── __pycache__
├── setuptools
└── setuptools-47.1.0.dist-info

6 directories, 1 file
(venv)

graalpython -m pip install pyrsistent
Collecting pyrsistent
  WARNING: Cache entry deserialization failed, entry ignored
  Downloading pyrsistent-0.18.0.tar.gz (104 kB)
     |████████████████████████████████| 104 kB 644 kB/s 
Looking for Graal Python patches for pyrsistent
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
ERROR: Exception:
Traceback (most recent call last):
  File "/home/ryuta/venv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 188, in _main
    status = self.run(options, args)
  File "/home/ryuta/venv/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 185, in wrapper
    return func(self, options, args)
  File "/home/ryuta/venv/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 332, in run
    requirement_set = resolver.resolve(
  File "/home/ryuta/venv/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 179, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
  File "/home/ryuta/venv/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 362, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/home/ryuta/venv/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 314, in _get_abstract_dist_for
    abstract_dist = self.preparer.prepare_linked_requirement(req)
  File "/home/ryuta/venv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 487, in prepare_linked_requirement
    abstract_dist = _get_prepared_distribution(
  File "/home/ryuta/venv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 91, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "/home/ryuta/venv/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 38, in prepare_distribution_metadata
    self._setup_isolation(finder)
  File "/home/ryuta/venv/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 96, in _setup_isolation
    reqs = backend.get_requires_for_build_wheel()
  File "/home/ryuta/venv/lib/python3.8/site-packages/pip/_vendor/pep517/wrappers.py", line 160, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
  File "/home/ryuta/venv/lib/python3.8/site-packages/pip/_vendor/pep517/wrappers.py", line 265, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
  File "/home/ryuta/venv/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 86, in _build_backend
    obj = import_module(mod_path)
  File "/home/ryuta/packages/graalvm/graal/graal/sdk/mxbuild/linux-amd64/GRAALVM_8052F32AFF_JAVA11/graalvm-8052f32aff-java11-21.3.0-dev/languages/python/lib-python/3/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'setuptools'

WARNING: You are using pip version 20.1.1; however, version 21.1.3 is available.
You should consider upgrading via the '/home/ryuta/local/graal/languages/python/bin/graalpython -m pip install --upgrade pip' command.
(venv)

[msimacek]

I cannot reproduce this. Are you using current master branch? Could you please post all the commands that you execute including the virtualenv creation?

The last line suggests that the graalpython executable might not be the one from virtualenv. Maybe your shell didn't pick up the updated PATH after the virtualenv was activated? Can you try running hash -r in that shell and retry the command?

[oroppas]

Sorry. My environment was messed up. I followed the instruction on https://github.com/oracle/graalpython to create and activate venv. I needed to install wheel

./venv/bin/graalpython -m pip install wheel
Collecting wheel
  WARNING: Cache entry deserialization failed, entry ignored
  Downloading wheel-0.36.2-py2.py3-none-any.whl (35 kB)
Looking for Graal Python patches for wheel
We have patches to make this package work on GraalVM for some version(s).
If installing or running fails, consider using one of the versions that we have patches for:
    wheel-0.33.patch
    wheel-0.34.patch
Installing collected packages: wheel
Looking for Graal Python patches for wheel
We have patches to make this package work on GraalVM for some version(s).
If installing or running fails, consider using one of the versions that we have patches for:
    wheel-0.33.patch
    wheel-0.34.patch
Successfully installed wheel-0.36.2
WARNING: You are using pip version 20.1.1; however, version 21.1.3 is available.
You should consider upgrading via the '/home/ryuta/venv/bin/graalpython -m pip install --upgrade pip' command.
(venv)

before I install pyrsistent

./venv/bin/graalpython -m pip install pyrsistent
Collecting pyrsistent
  WARNING: Cache entry deserialization failed, entry ignored
  Downloading pyrsistent-0.18.0.tar.gz (104 kB)
     |████████████████████████████████| 104 kB 383 kB/s 
Looking for Graal Python patches for pyrsistent
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
    Preparing wheel metadata ... done
Building wheels for collected packages: pyrsistent
  Building wheel for pyrsistent (PEP 517) ... done
  Created wheel for pyrsistent: filename=pyrsistent-0.18.0-py3-none-any.whl size=56321 sha256=aed7dedb2212b94ea9b52ee00657be7d88ef2f5d4807583005552bc5a0c66406
  Stored in directory: /home/ryuta/.cache/pip/wheels/d9/d1/94/8c9af2be5e7dba996dcd7c2eee9635f947d02e89c05ebc5fb4
Successfully built pyrsistent
Installing collected packages: pyrsistent
Looking for Graal Python patches for pyrsistent
Successfully installed pyrsistent-0.18.0
WARNING: You are using pip version 20.1.1; however, version 21.1.3 is available.
You should consider upgrading via the '/home/ryuta/venv/bin/graalpython -m pip install --upgrade pip' command.
(venv)

[oroppas]

Now I can confirm graalpython can install numpy under venv 🚀

Make sure to install Cython before installing numpy

./venv/bin/graalpython -m pip install Cython
Collecting Cython
  WARNING: Cache entry deserialization failed, entry ignored
  Downloading Cython-0.29.24-py2.py3-none-any.whl (979 kB)
     |████████████████████████████████| 979 kB 28 kB/s 
Looking for Graal Python patches for Cython
Installing collected packages: Cython
Looking for Graal Python patches for Cython
Successfully installed Cython-0.29.24
WARNING: You are using pip version 20.1.1; however, version 21.1.3 is available.
You should consider upgrading via the '/home/ryuta/venv/bin/graalpython -m pip install --upgrade pip' command.
(venv)

then

./venv/bin/graalpython -m pip install numpy
Collecting numpy
  WARNING: Cache entry deserialization failed, entry ignored
  Downloading numpy-1.21.1.zip (10.3 MB)
     |████████████████████████████████| 10.3 MB 23 kB/s 
Looking for Graal Python patches for numpy
We have patches to make this package work on GraalVM for some version(s).
If installing or running fails, consider using one of the versions that we have patches for:
    numpy-1.16.4.patch
  Installing build dependencies ... done
  WARNING: Missing build requirements in pyproject.toml for numpy from https://files.pythonhosted.org/packages/0b/a7/e724c8df240687b5fd62d8c71f1a6709d455c4c09432c7412e3e64f4cbe5/numpy-1.21.1.zip#sha256=dff4af63638afcc57a3dfb9e4b26d434a7a602d225b42d746ea7fe2edf1342fd.
  WARNING: The project does not specify a build backend, and pip cannot fall back to setuptools without 'setuptools>=40.8.0' and 'wheel'.
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
    Preparing wheel metadata ... done
Building wheels for collected packages: numpy
  Building wheel for numpy (PEP 517) ... done
  Created wheel for numpy: filename=numpy-1.21.1-graalpython38-graalpython_38_native_x86_64_linux-linux_i686.whl size=29530985 sha256=0e21cb09db266759692c674dc20969bcdec3b98e340edcd7a5c68e0bc48a58c7
  Stored in directory: /home/ryuta/.cache/pip/wheels/96/6f/59/eb8d9d8f401c341585fe50477183f1762db5f71b1c464f8b03
Successfully built numpy
Installing collected packages: numpy
Looking for Graal Python patches for numpy
Successfully installed numpy-1.21.1
WARNING: You are using pip version 20.1.1; however, version 21.1.3 is available.
You should consider upgrading via the '/home/ryuta/venv/bin/graalpython -m pip install --upgrade pip' command.
(venv)

[oroppas]

Before I close this issue, can you help me understand why graalpython -m pip install pyrsisten does not work outside of venv?

Thanks!

[msimacek]

You mean why you had to use the ./venv/bin/graalpython instead of just graalpython? Not sure, if you activated it, it should just work with both just graalpython or python. If you want to debug it further, with the virtualenv active, what's the output of echo $PATH, which graalpython and hash -t graalpython? And what's your shell?

Or why installing outside a virtualenv doesn't work? The layout of a GraalVM distribution is different from CPython's. We should be able to adapt the configuration to make it work. It just wasn't on our priority list, since you can do everything with virtualenvs.

[oroppas]

Or why installing outside a virtualenv doesn't work? The layout of a GraalVM distribution is different from CPython's. We should be able to adapt the configuration to make it work. It just wasn't on our priority list, since you can do everything with virtualenvs.

Make sense. Thanks!