search

NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
17.43k stars 13.64k forks source link

pypy.withPackages can’t import packages #39356

Open andersk opened 6 years ago

andersk commented 6 years ago

Issue description

The pypy from, e.g., pypy.withPackages (ps: [ps.setuptools]) can’t import setuptools. Same for every other package I’ve tried. (Split off from #34066.)

Steps to reproduce

$ nix-shell --pure -p 'pypy.withPackages (ps: [ps.setuptools])' --run 'pypy -c "import setuptools"'
Traceback (most recent call last):
  File "<module>", line 1, in <module>
ImportError: No module named setuptools

Technical details

FRidh commented 6 years ago

That's likely related to (not) updating the setup hook.

andersk commented 6 years ago

If you mean pkgs/development/interpreters/python/setup-hook.sh, I don’t think that’s involved, as none of DETERMINISTIC_BUILD, PYTHONHASHSEED, nor PYTHONNOUSERSITE are set in either the non-working nix-shell --pure -p 'pypy.withPackages (ps: [ps.setuptools])' or the working nix-shell --pure -p 'python.withPackages (ps: [ps.setuptools])'.

FRidh commented 5 years ago

For now, consider using pythonInterpreters.pypy_prebuilt or pythonInterpreters.pypy3_prebuilt. These use patchelfed prebuilt binaries. I am quite sure these work. Unfortunately, fixing the from source builds always takes a lot of time. Note that these prebuilt ones can be used to speed up the building/translation of the other ones.

andersk commented 5 years ago

@FRidh Same problem with the prebuilt versions.

$ nix-shell --pure -p 'pythonInterpreters.pypy27_prebuilt.withPackages (ps: [ps.setuptools])' --run 'pypy -c "import setuptools"'
/nix/store/wciajzl7c5r76j32l8aj265bw7pdrqwf-pypy_prebuilt-6.0.0/bin/pypy: /nix/store/kimqdijh6r0p8d6if8xacnijl2y4ngw5-openssl-1.0.2q/lib/libssl.so.1.0.0: no version information available (required by /nix/store/wciajzl7c5r76j32l8aj265bw7pdrqwf-pypy_prebuilt-6.0.0/lib/libpypy-c.so)
/nix/store/wciajzl7c5r76j32l8aj265bw7pdrqwf-pypy_prebuilt-6.0.0/bin/pypy: /nix/store/kimqdijh6r0p8d6if8xacnijl2y4ngw5-openssl-1.0.2q/lib/libssl.so.1.0.0: no version information available (required by /nix/store/wciajzl7c5r76j32l8aj265bw7pdrqwf-pypy_prebuilt-6.0.0/lib/libpypy-c.so)
/nix/store/wciajzl7c5r76j32l8aj265bw7pdrqwf-pypy_prebuilt-6.0.0/bin/pypy: /nix/store/kimqdijh6r0p8d6if8xacnijl2y4ngw5-openssl-1.0.2q/lib/libcrypto.so.1.0.0: no version information available (required by /nix/store/wciajzl7c5r76j32l8aj265bw7pdrqwf-pypy_prebuilt-6.0.0/lib/libpypy-c.so)
Traceback (most recent call last):
  File "<module>", line 1, in <module>
ImportError: No module named setuptools
FRidh commented 5 years ago

Let's fix those first then.

FRidh commented 5 years ago

Looks like pypy doesn't support PYTHONHOME. It's not documented, and setting it has no effect on sys.prefix or sys.exec_prefix. Also, the site-packages folder that is used when building packages is hardcoded at build-time it seems, which is pretty annoying. I think it would be good if pypy's sysconfig was more like that of cpython. However, during runtime, the site-packages folder on sys.path is elsewhere.

MostAwesomeDude commented 5 years ago

PyPy's sourcecode indicates in comments that PYTHONHOME is not supported.

FRidh commented 5 years ago

good that we're not using PYTHONHOME anymore but NIX_PYTHONPATH ;-) The latter is implemented in a sitecustomize.py

FRidh commented 5 years ago

The prebuilt ones function with Nixpkgs at f2c619fae8e2b82cde03c91bad45cf7f30b4841a:

$ nix run  "(import ./. {}).pythonInterpreters.pypy36_prebuilt.withPackages(ps: with ps; [ pytest ])" --command "pytest"
stale[bot] commented 4 years ago

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

  1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
  2. Ask on the NixOS Discourse.
  3. Ask on the #nixos channel on irc.freenode.net.
andersk commented 4 years ago

Still an issue.

mcncm commented 3 years ago

I've just encountered this problem, too. Is anyone looking into it any more?

mateon1 commented 2 years ago

Is any work on this being done? I'm currently working around this problem by manually setting PYTHONPATH, e.g.:

PYTHONPATH=/nix/store/z5c5hfmhglwm8gchs1qy00ghafrp3r7w-pypy2.7-purepng-0.2.0/site-packages/: pypy script.py

This is far from ideal, as it's rather difficult to find the site-packages paths for this instance of pypy.withPackages that was built.