pypy derivations produce broken python installation for x86_64-linux

[jab]

Issue description

The x86_64-linux pypy installations provided by the pypy3.10 and pypy3.9 derivations are broken such that python -m sysconfig (or equivalently pypy3 -m sysconfig) fails with:

ModuleNotFoundError: No module named '_sysconfigdata__linux_x86_64-linux-gnu'

This causes lots of other functionality in the standard library to fail, e.g. python -m venv foo, python -m ensurepip, etc.

(Click to toggle example trace from a GitHub Actions 'ubuntu-latest' runner)

``` (nix:nix-shell-env) runner@fv-az1498-25:~$ pypy3.10 -VV Python 3.10.12 (af44d0b8114cb82c40a07bb9ee9c1ca8a1b3688c, Jan 03 2024, 06:33:26) [PyPy 7.3.12 with GCC 13.2.0] (nix:nix-shell-env) runner@fv-az1498-25:~$ pypy3.10 -m sysconfig Platform: "linux-x86_64" Python version: "3.10" Current installation scheme: "posix_prefix" Traceback (most recent call last): File "/nix/store/m9iaqdg9qhj2qja69g0ni103ds6dax10-pypy3.10-7.3.12/pypy3.10-c/lib-python/3/runpy.py", line 199, in _run_module_as_main return _run_code(code, main_globals, None, File "/nix/store/m9iaqdg9qhj2qja69g0ni103ds6dax10-pypy3.10-7.3.12/pypy3.10-c/lib-python/3/runpy.py", line 86, in _run_code exec(code, run_globals) File "/nix/store/m9iaqdg9qhj2qja69g0ni103ds6dax10-pypy3.10-7.3.12/pypy3.10-c/lib-python/3/sysconfig.py", line 832, in _main() File "/nix/store/m9iaqdg9qhj2qja69g0ni103ds6dax10-pypy3.10-7.3.12/pypy3.10-c/lib-python/3/sysconfig.py", line 826, in _main _print_dict('Paths', get_paths()) ^^^^^^^^^^^ File "/nix/store/m9iaqdg9qhj2qja69g0ni103ds6dax10-pypy3.10-7.3.12/pypy3.10-c/lib-python/3/sysconfig.py", line 586, in get_paths return _expand_vars(scheme, vars) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/nix/store/m9iaqdg9qhj2qja69g0ni103ds6dax10-pypy3.10-7.3.12/pypy3.10-c/lib-python/3/sysconfig.py", line 225, in _expand_vars _extend_dict(vars, get_config_vars()) ^^^^^^^^^^^^^^^^^ File "/nix/store/m9iaqdg9qhj2qja69g0ni103ds6dax10-pypy3.10-7.3.12/pypy3.10-c/lib-python/3/sysconfig.py", line 642, in get_config_vars _init_posix(_CONFIG_VARS) File "/nix/store/m9iaqdg9qhj2qja69g0ni103ds6dax10-pypy3.10-7.3.12/pypy3.10-c/lib-python/3/sysconfig.py", line 498, in _init_posix _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ModuleNotFoundError: No module named '_sysconfigdata__linux_x86_64-linux-gnu' (nix:nix-shell-env) runner@fv-az1498-25:~$ echo $? 1 ```

Steps to reproduce

Locally I can only test on 'aarch64-darwin' and this does not occur there, but I consistently reproduce this in GitHub Actions' 'ubuntu-latest' runners (which are x86_64) with Nix installed via https://github.com/DeterminateSystems/nix-installer-action, by running the following in an SSH session:

1. nix-shell -p pypy3.10 -p pypy3.9
2. pypy3.10 -m sysconfig
3. pypy3.9 -m sysconfig

(Here is an associated `flake.lock` as well)

```json { "nodes": { "nixpkgs": { "locked": { "lastModified": 1705175803, "narHash": "sha256-u+WX/r8j+jKef3IAVNWW8fznj7pn9kkaMtzjten2Qec=", "owner": "NixOS", "repo": "nixpkgs", "rev": "2284e61bd9df081fb2d4cc162b1d6b3d9752b02d", "type": "github" }, "original": { "owner": "NixOS", "repo": "nixpkgs", "type": "github" } }, "root": { "inputs": { "nixpkgs": "nixpkgs" } } }, "root": "root", "version": 7 } ```

Technical details

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

 - system: `"x86_64-linux"`
 - host os: `Linux 6.2.0-1018-azure, Ubuntu, 22.04.3 LTS (Jammy Jellyfish), nobuild`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.1`
 - nixpkgs: `/nix/store/g16z4fs1mrbkxc4x6wm8xbrh13nc7aw4-source`

[kirillrdy]

hi, I cant reproduce the issue on nixos

 git checkout 2284e61bd9df081fb2d4cc162b1d6b3d9752b02d
 nix-build -A pypy310
./result/bin/pypy3 -m sysconfig

Platform: "linux-x86_64"
Python version: "3.10"
Current installation scheme: "posix_prefix"

Paths: 
    data = "/home/kirillvr/nixpkgs/result/pypy3.10-c"
    include = "/home/kirillvr/nixpkgs/result/pypy3.10-c/include/pypy3.10"
    platinclude = "/home/kirillvr/nixpkgs/result/pypy3.10-c/include/pypy3.10"
    platlib = "/home/kirillvr/nixpkgs/result/pypy3.10-c/lib/pypy3.10/site-packages"
    platstdlib = "/home/kirillvr/nixpkgs/result/pypy3.10-c/lib/pypy3.10"
    purelib = "/home/kirillvr/nixpkgs/result/pypy3.10-c/lib/pypy3.10/site-packages"
    scripts = "/home/kirillvr/nixpkgs/result/pypy3.10-c/bin"
    stdlib = "/home/kirillvr/nixpkgs/result/pypy3.10-c/lib/pypy3.10"

[jab]

Thanks for testing. Can you share your nix-shell -p nix-info --run "nix-info -m" output, and is there any chance you're getting different results due to a difference there?

[kirillrdy]

nix-info -m
 - system: `"x86_64-linux"`
 - host os: `Linux 6.6.11, NixOS, 24.05 (Uakari), 24.05.20240113.7e6aa7e`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.1`
 - channels(kirillvr): `""`
 - channels(root): `"nixos"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[undefined-moe]

It's now

nix-repl> pypy3
error:
       … while calling the 'derivationStrict' builtin

         at /builtin/derivation.nix:9:12: (source not available)

       … while evaluating derivation 'pypy3.9-7.3.12'
         whose name attribute is located at /nix/store/pw3hllgllk54qir2zvahrdb0a5rrmih4-source/pkgs/stdenv/generic/make-derivation.nix:331:7

       … while evaluating attribute 'C_INCLUDE_PATH' of derivation 'pypy3.9-7.3.12'

         at /nix/store/pw3hllgllk54qir2zvahrdb0a5rrmih4-source/pkgs/development/interpreters/python/pypy/default.nix:74:3:

           73|
           74|   C_INCLUDE_PATH = lib.makeSearchPathOutput "dev" "include" buildInputs;
             |   ^
           75|   LIBRARY_PATH = lib.makeLibraryPath buildInputs;

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: pycparser-2.22 not supported for interpreter pypy
«derivation 

on rev ac34158a823c7596e0106c806d0b7df47885fa73.

[wrdls]

Seeing a similar error as @undefined-moe on mac os.

❯ nix-shell -p pypy
error:
       … while calling the 'derivationStrict' builtin

         at /builtin/derivation.nix:9:12: (source not available)

       … while evaluating derivation 'shell'
         whose name attribute is located at /nix/store/y4p1bzzriaprvvy6fd9nz3kifjdiblq8-source/pkgs/stdenv/generic/make-derivation.nix:331:7

       … while evaluating attribute 'buildInputs' of derivation 'shell'

         at /nix/store/y4p1bzzriaprvvy6fd9nz3kifjdiblq8-source/pkgs/stdenv/generic/make-derivation.nix:378:7:

          377|       depsHostHost                = elemAt (elemAt dependencies 1) 0;
          378|       buildInputs                 = elemAt (elemAt dependencies 1) 1;
             |       ^
          379|       depsTargetTarget            = elemAt (elemAt dependencies 2) 0;

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: pycparser-2.22 not supported for interpreter pypy

❯ nix-shell -p nix-info --run "nix-info -m"
these 3 paths will be fetched (1.03 MiB download, 8.17 MiB unpacked):
  /nix/store/598vvx2b3lk4p6gh90329mv0fh0a4inm-DarwinTools-1
  /nix/store/x2hwnm42a41wgf8zk53aay78vrklrsxq-bash-interactive-5.2p26
  /nix/store/1ysw4h9l4gclh46sk1jmvsh8jxjnwmk9-nix-info
copying path '/nix/store/x2hwnm42a41wgf8zk53aay78vrklrsxq-bash-interactive-5.2p26' from 'https://cache.nixos.org'...
copying path '/nix/store/598vvx2b3lk4p6gh90329mv0fh0a4inm-DarwinTools-1' from 'https://cache.nixos.org'...
copying path '/nix/store/1ysw4h9l4gclh46sk1jmvsh8jxjnwmk9-nix-info' from 'https://cache.nixos.org'...
 - system: `"aarch64-darwin"`
 - host os: `Darwin 23.4.0, macOS 14.4.1`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.18.1`
 - nixpkgs: `/nix/store/y4p1bzzriaprvvy6fd9nz3kifjdiblq8-source`