pdm: fails to create virtualenv

qsuscs commented 1 year ago

Describe the bug

When running pdm install in a PDM-managed Python project, with python.use_venv on (which is its default), it fails to create a virtualenv. This is because it calls /nix/store/…-python3-…/bin/python3.… -m virtualenv, which does not have the required paths in sys.path and hence fails to find virtualenv.

Steps To Reproduce

Steps to reproduce the behavior:

git clone https://github.com/pdm-project/pdm-example-monorepo
cd pdm-example-monorepo
git checkout a6b3f47027730829105fae6524e4096d1ac180e4
pdm install

Expected behavior

PDM should be able to create a virtualenv.

~Screenshots~ Terminal log

thomas@hoth /tmp/pdm-example-monorepo (git)-[main]
% pdm install
Lock file version is not compatible with PDM, installation may fail
Updating the lock file...
python.use_venv is on, creating a virtualenv for this project...
/nix/store/a4ivjgjcdk0l4yzylhc1q7acm7abdz6x-python3-3.10.10/bin/python3.10: No module named virtualenv
[VirtualenvCreateError]: Command
'['/nix/store/a4ivjgjcdk0l4yzylhc1q7acm7abdz6x-python3-3.10.10/bin/python3.10', '-m',
'virtualenv', '/private/tmp/pdm-example-monorepo/.venv', '-p',
'/nix/store/a4ivjgjcdk0l4yzylhc1q7acm7abdz6x-python3-3.10.10/bin/python', '--no-pip',
'--no-setuptools', '--no-wheel', '--prompt=pdm-example-monorepo-3.10']' returned
non-zero exit status 1.

Additional context

This is with pdm from nixpkgs-unstable (the version in 22.11 is too old for my project’s metadata format version), but I don’t think this is the cause for this bug. I wanted to make this clear by using the correct nix-shell command above for reproducing it, but failed at the correct incantation.

Notify maintainers

@cpcloud

Metadata

thomas@hoth /tmp
% nix-shell -p nix-info --run "nix-info -m"
these 2 paths will be fetched (0.01 MiB download, 0.03 MiB unpacked):
  /nix/store/f808fny98wqvjf0i57ibgl53ayshmdss-DarwinTools-1
  /nix/store/i3gkim5qjs0zhcjv3nx4ns93yjq21zjb-nix-info
copying path '/nix/store/f808fny98wqvjf0i57ibgl53ayshmdss-DarwinTools-1' from 'https://cache.nixos.org'...
copying path '/nix/store/i3gkim5qjs0zhcjv3nx4ns93yjq21zjb-nix-info' from 'https://cache.nixos.org'...
 - system: `"x86_64-darwin"`
 - host os: `Darwin 19.6.0, macOS 10.15.7`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.12.0`
 - channels(root): `"nixpkgs-22.11-darwin"`
 - channels(thomas): `"home-manager-22.11.tar.gz, nixpkgs-unstable"`
 - nixpkgs: `/Users/thomas/.nix-defexpr/channels_root/nixpkgs`

heshdotcc commented 1 year ago

Here, same issue:

% nix-shell -p nix-info --run "nix-info -m"
Python 3.11.1
 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.93, NixOS, 23.05 (Stoat), 23.05.20230211.e5530ab`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.13.2`
 - channels(root): `"home-manager, nixos"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

natsukium commented 1 year ago

I also had this problem when I went back to venv from PEP582.

It seems to be related to #66366. The virtualenv is included in the propagatedBuildInputs but not in the site-packages of the Python environment.

A possible workaround is to use venv.backend = venv in the configuration of pdm, or give it an argument like pdm venv create -w venv. https://pdm.fming.dev/latest/reference/configuration/#available-configurations

It would be easier for the package to patch the configuration to default to venv until the above issue is resolved.

cc @cpcloud @mweinelt @betaboon

NixOS / nixpkgs