search

cachix / devenv

Fast, Declarative, Reproducible, and Composable Developer Environments
https://devenv.sh
Apache License 2.0
3.56k stars 259 forks source link

Python language enable breaks devenv #555

Closed jocelynthode closed 1 month ago

jocelynthode commented 1 year ago

Describe the bug When activating languages.python.enable = true devenv up returns 

/nix/store/iw1vmh509hcbby8dbpsaanbri4zsq7dj-python3-3.10.10/bin/python: /nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224/lib/libc.so.6: version `GLIBC_2.36' not found (required by /nix/store/wy0incigsdz3nai26lxmn9ibchnb0qd6-libxcrypt-4.4.33/lib/libcrypt.so.2)

Installing glibc via devenv just results in a core dump.

This is probably because the python version from the languages overrides my system python.

Not sure how I could still have devenv working while having python enabled

To reproduce https://gist.github.com/jocelynthode/f97bc7a966d4e19ca50fc1ee4551d014

Version

devenv: 0.6.2

domenkozar commented 1 year ago

We just merged a fix in the master, can you try devenv update?

jocelynthode commented 1 year ago

@domenkozar: I'm still experiencing the same issue with this devenv.lock:

{
  "nodes": {
    "devenv": {
      "locked": {
        "dir": "src/modules",
        "lastModified": 1682503657,
        "narHash": "sha256-/YAlnySFwbtWJarI0/iGLUGa8FAAItbaB4HhiVGIZa8=",
        "owner": "cachix",
        "repo": "devenv",
        "rev": "9465ba9ca1ec1e73d139f36b1f96b094163d3ffc",
        "type": "github"
      },
      "original": {
        "dir": "src/modules",
        "owner": "cachix",
        "repo": "devenv",
        "type": "github"
      }
    },
    "flake-compat": {
      "flake": false,
      "locked": {
        "lastModified": 1673956053,
        "narHash": "sha256-4gtG9iQuiKITOjNQQeQIpoIB6b16fm+504Ch3sNKLd8=",
        "owner": "edolstra",
        "repo": "flake-compat",
        "rev": "35bb57c0c8d8b62bbfd284272c928ceb64ddbde9",
        "type": "github"
      },
      "original": {
        "owner": "edolstra",
        "repo": "flake-compat",
        "type": "github"
      }
    },
    "flake-utils": {
      "locked": {
        "lastModified": 1667395993,
        "narHash": "sha256-nuEHfE/LcWyuSWnS8t12N1wc105Qtau+/OdUAjtQ0rA=",
        "owner": "numtide",
        "repo": "flake-utils",
        "rev": "5aed5285a952e0b949eb3ba02c12fa4fcfef535f",
        "type": "github"
      },
      "original": {
        "owner": "numtide",
        "repo": "flake-utils",
        "type": "github"
      }
    },
    "gitignore": {
      "inputs": {
        "nixpkgs": [
          "pre-commit-hooks",
          "nixpkgs"
        ]
      },
      "locked": {
        "lastModified": 1660459072,
        "narHash": "sha256-8DFJjXG8zqoONA1vXtgeKXy68KdJL5UaXR8NtVMUbx8=",
        "owner": "hercules-ci",
        "repo": "gitignore.nix",
        "rev": "a20de23b925fd8264fd7fad6454652e142fd7f73",
        "type": "github"
      },
      "original": {
        "owner": "hercules-ci",
        "repo": "gitignore.nix",
        "type": "github"
      }
    },
    "nixpkgs": {
      "locked": {
        "lastModified": 1682109806,
        "narHash": "sha256-d9g7RKNShMLboTWwukM+RObDWWpHKaqTYXB48clBWXI=",
        "owner": "NixOS",
        "repo": "nixpkgs",
        "rev": "2362848adf8def2866fabbffc50462e929d7fffb",
        "type": "github"
      },
      "original": {
        "owner": "NixOS",
        "ref": "nixpkgs-unstable",
        "repo": "nixpkgs",
        "type": "github"
      }
    },
    "nixpkgs-stable": {
      "locked": {
        "lastModified": 1678872516,
        "narHash": "sha256-/E1YwtMtFAu2KUQKV/1+KFuReYPANM2Rzehk84VxVoc=",
        "owner": "NixOS",
        "repo": "nixpkgs",
        "rev": "9b8e5abb18324c7fe9f07cb100c3cd4a29cda8b8",
        "type": "github"
      },
      "original": {
        "owner": "NixOS",
        "ref": "nixos-22.11",
        "repo": "nixpkgs",
        "type": "github"
      }
    },
    "pre-commit-hooks": {
      "inputs": {
        "flake-compat": "flake-compat",
        "flake-utils": "flake-utils",
        "gitignore": "gitignore",
        "nixpkgs": [
          "nixpkgs"
        ],
        "nixpkgs-stable": "nixpkgs-stable"
      },
      "locked": {
        "lastModified": 1682326782,
        "narHash": "sha256-wj7p7iEwQXAfTZ6QokAe0dMbpQk5u7ympDnaiPvbv1w=",
        "owner": "cachix",
        "repo": "pre-commit-hooks.nix",
        "rev": "56cd2d47a9c937be98ab225cf014b450f1533cdb",
        "type": "github"
      },
      "original": {
        "owner": "cachix",
        "repo": "pre-commit-hooks.nix",
        "type": "github"
      }
    },
    "root": {
      "inputs": {
        "devenv": "devenv",
        "nixpkgs": "nixpkgs",
        "pre-commit-hooks": "pre-commit-hooks"
      }
    }
  },
  "root": "root",
  "version": 7
}
domenkozar commented 1 year ago

What if you delete .devenv/state in your project?

jocelynthode commented 1 year ago

Unfortunately I get the exact same error.

domenkozar commented 1 year ago

What OS/distro?

jocelynthode commented 1 year ago

I'm using NixOS unstable. Here is the flake.nix I use to build my system: https://github.com/jocelynthode/nixos-config/blob/master/flake.nix

When copying the devenv.nix, devenv.lock, devenv.yaml and .envrc to an empty folder I can reproduce the issue.

bobvanderlinden commented 1 year ago

This seemed very similar to the error that https://github.com/cachix/devenv/pull/552 attempted to solve.

I tried to reproduce this by cloning the gist, but I it worked fine here. I also tried to place a failing python binary into PATH before calling devenv shell, but it seems that isn't being called. I can imagine it might have something to do with the VIRTUAL_ENV environment variable being there already (before devenv shell is started), but am not sure.

As a response to this issue I thought it might be nice to debug such issues more in-depth: https://github.com/cachix/devenv/pull/556

jocelynthode commented 1 year ago

@bobvanderlinden: Thanks for trying to replicate.If you cannot then it might be a problem on my end rather than with devenv. I've added your PR in my devenv.yaml to be able to test with debug and here's the output: https://gist.github.com/jocelynthode/82796a32f7ae1e37b7951f58b0b89acb

jocelynthode commented 1 year ago

As a hunch I've tried specifying languages.python.package = pkgs.python311; and it completely solves the issue. So the issue seemsto only appear when both devenv and my NixOS system have the same python version.

bobvanderlinden commented 1 year ago

@bobvanderlinden: Thanks for trying to replicate.If you cannot then it might be a problem on my end rather than with devenv. I've added your PR in my devenv.yaml to be able to test with debug and here's the output: https://gist.github.com/jocelynthode/82796a32f7ae1e37b7951f58b0b89acb

Oo nice, thanks! Good to have a concrete test-case for the PR as well :smile::+1:

From the Gist it seems that devenv is correctly creating a shell and exiting with 0. Does it only go wrong when calling devenv up? Does devenv up show anything else in its output?

jocelynthode commented 1 year ago

It goes wrong when calling devenv up or devenv info. Nothing else is outputted.

bobvanderlinden commented 1 year ago

What is the output of the following?

bash -x $(command -v devenv) info
jocelynthode commented 1 year ago

Here is the result:

+ set -e
+ NIX_FLAGS='--show-trace --extra-experimental-features nix-command --extra-experimental-features flakes --option warn-dirty false'
+ export FLAKE_FILE=.devenv.flake.nix
+ FLAKE_FILE=.devenv.flake.nix
+ export FLAKE_LOCK=devenv.lock
+ FLAKE_LOCK=devenv.lock
+ CUSTOM_NIX=/nix/store/iwl4jmplwfvxww7x1gsr5ak5x5hs2msr-nix-2.12.0pre20230216_7c91803
+ [[ -z /home/jocelyn/.local/share ]]
+ GC_ROOT=/home/jocelyn/.local/share/devenv/gc
+ mkdir -p /home/jocelyn/.local/share/devenv/gc
++ date +%s
+ GC_DIR=/home/jocelyn/.local/share/devenv/gc/1682523186
+ command=info
+ [[ ! -z info ]]
+ shift
+ case $command in
+ assemble
+ [[ ! -f devenv.nix ]]
++ pwd
+ export DEVENV_DIR=/tmp/devenv/.devenv
+ DEVENV_DIR=/tmp/devenv/.devenv
+ export DEVENV_GC=/tmp/devenv/.devenv/gc
+ DEVENV_GC=/tmp/devenv/.devenv/gc
+ mkdir -p /tmp/devenv/.devenv/gc
+ [[ -f devenv.yaml ]]
+ /nix/store/j5f9qc3v4f9mbb6ci4vnharxwac199ik-devenv-yaml/bin/devenv-yaml /tmp/devenv/.devenv
/nix/store/iw1vmh509hcbby8dbpsaanbri4zsq7dj-python3-3.10.10/bin/python: /nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224/lib/libc.so.6: version `GLIBC_2.36' not found (required by /nix/store/wy0incigsdz3nai26lxmn9ibchnb0qd6-libxcrypt-4.4.33/lib/libcrypt.so.2)
bobvanderlinden commented 1 year ago

Hmm, so devenv does supply the correct python version, but it is built against an incompatible glibc version? Just calling /nix/store/iw1vmh509hcbby8dbpsaanbri4zsq7dj-python3-3.10.10/bin/python should just work regardless of the host OS (or its glibc version) I'd think...

I have the same python3-3.10.10 locally. It runs ok and is linked against:

$ ldd /nix/store/iw1vmh509hcbby8dbpsaanbri4zsq7dj-python3-3.10.10/bin/python
    linux-vdso.so.1 (0x00007fff62f7e000)
    libpython3.10.so.1.0 => /nix/store/iw1vmh509hcbby8dbpsaanbri4zsq7dj-python3-3.10.10/lib/libpython3.10.so.1.0 (0x00007fd81f000000)
    libcrypt.so.1 => /nix/store/c06np1spdb2bbsfp5x3716d529mrxw7b-libxcrypt-4.4.33/lib/libcrypt.so.1 (0x00007fd81f589000)
    libdl.so.2 => /nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224/lib/libdl.so.2 (0x00007fd81f584000)
    libm.so.6 => /nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224/lib/libm.so.6 (0x00007fd81f4a4000)
    libgcc_s.so.1 => /nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224/lib/libgcc_s.so.1 (0x00007fd81f48a000)
    libc.so.6 => /nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224/lib/libc.so.6 (0x00007fd81ec00000)
    /nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224/lib/ld-linux-x86-64.so.2 => /nix/store/1n2l5law9g3b77hcfyp50vrhhssbrj5g-glibc-2.37-8/lib64/ld-linux-x86-64.so.2 (0x00007fd81f5c7000)

It is strange that your error mentions /nix/store/wy0incigsdz3nai26lxmn9ibchnb0qd6-libxcrypt-4.4.33/lib/libcrypt.so.2 whereas mine is using a different libxcrypt. Not sure how that could happen :confused: Also not sure if that is part of the problem. Though I must admit this gets a bit in the 'out of my league'-terratory :sweat_smile:

@domenkozar any idea?

tfmoraes commented 1 year ago

If I pkgs.gcc-unwrapped.lib is added to the packages (because of pytorch, for instance) this error happens:

❯ devenv info
/nix/store/b38iyffa5xsd26id1400b404c29p3s7d-nix-2.12.0pre20230216_7c91803/bin/nix: /nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224/lib/libc.so.6: version `GLIBC_2.36' not found (required by /nix/store/r2pr6vgfhdb9mch8csj4xva20jrb73rv-devenv-profile/lib/libstdc++.so.6)

My devenv.nix:

{ pkgs, ... }:

{
  env.GREET = "devenv";

  packages = [
    pkgs.git
    pkgs.gcc-unwrapped.lib
  ];

  languages.python.enable = true;
  languages.python.package = pkgs.python311;
  languages.python.venv.enable = true;
}

And devenv.lock:

{
  "nodes": {
    "devenv": {
      "locked": {
        "dir": "src/modules",
        "lastModified": 1682674748,
        "narHash": "sha256-HQTp0U+OAlbH8Q7WBZglVA09mRNectkM17NZs4ebW3A=",
        "owner": "cachix",
        "repo": "devenv",
        "rev": "971ca55b91f3f326b7067ec92345e4527fc01805",
        "type": "github"
      },
      "original": {
        "dir": "src/modules",
        "owner": "cachix",
        "repo": "devenv",
        "type": "github"
      }
    },
    "flake-compat": {
      "flake": false,
      "locked": {
        "lastModified": 1673956053,
        "narHash": "sha256-4gtG9iQuiKITOjNQQeQIpoIB6b16fm+504Ch3sNKLd8=",
        "owner": "edolstra",
        "repo": "flake-compat",
        "rev": "35bb57c0c8d8b62bbfd284272c928ceb64ddbde9",
        "type": "github"
      },
      "original": {
        "owner": "edolstra",
        "repo": "flake-compat",
        "type": "github"
      }
    },
    "flake-utils": {
      "locked": {
        "lastModified": 1667395993,
        "narHash": "sha256-nuEHfE/LcWyuSWnS8t12N1wc105Qtau+/OdUAjtQ0rA=",
        "owner": "numtide",
        "repo": "flake-utils",
        "rev": "5aed5285a952e0b949eb3ba02c12fa4fcfef535f",
        "type": "github"
      },
      "original": {
        "owner": "numtide",
        "repo": "flake-utils",
        "type": "github"
      }
    },
    "gitignore": {
      "inputs": {
        "nixpkgs": [
          "pre-commit-hooks",
          "nixpkgs"
        ]
      },
      "locked": {
        "lastModified": 1660459072,
        "narHash": "sha256-8DFJjXG8zqoONA1vXtgeKXy68KdJL5UaXR8NtVMUbx8=",
        "owner": "hercules-ci",
        "repo": "gitignore.nix",
        "rev": "a20de23b925fd8264fd7fad6454652e142fd7f73",
        "type": "github"
      },
      "original": {
        "owner": "hercules-ci",
        "repo": "gitignore.nix",
        "type": "github"
      }
    },
    "nixpkgs": {
      "locked": {
        "lastModified": 1682526928,
        "narHash": "sha256-2cKh4O6t1rQ8Ok+v16URynmb0rV7oZPEbXkU0owNLQs=",
        "owner": "NixOS",
        "repo": "nixpkgs",
        "rev": "d6b863fd9b7bb962e6f9fdf292419a775e772891",
        "type": "github"
      },
      "original": {
        "owner": "NixOS",
        "ref": "nixos-unstable",
        "repo": "nixpkgs",
        "type": "github"
      }
    },
    "nixpkgs-stable": {
      "locked": {
        "lastModified": 1678872516,
        "narHash": "sha256-/E1YwtMtFAu2KUQKV/1+KFuReYPANM2Rzehk84VxVoc=",
        "owner": "NixOS",
        "repo": "nixpkgs",
        "rev": "9b8e5abb18324c7fe9f07cb100c3cd4a29cda8b8",
        "type": "github"
      },
      "original": {
        "owner": "NixOS",
        "ref": "nixos-22.11",
        "repo": "nixpkgs",
        "type": "github"
      }
    },
    "pre-commit-hooks": {
      "inputs": {
        "flake-compat": "flake-compat",
        "flake-utils": "flake-utils",
        "gitignore": "gitignore",
        "nixpkgs": [
          "nixpkgs"
        ],
        "nixpkgs-stable": "nixpkgs-stable"
      },
      "locked": {
        "lastModified": 1682596858,
        "narHash": "sha256-Hf9XVpqaGqe/4oDGr30W8HlsWvJXtMsEPHDqHZA6dDg=",
        "owner": "cachix",
        "repo": "pre-commit-hooks.nix",
        "rev": "fb58866e20af98779017134319b5663b8215d912",
        "type": "github"
      },
      "original": {
        "owner": "cachix",
        "repo": "pre-commit-hooks.nix",
        "type": "github"
      }
    },
    "root": {
      "inputs": {
        "devenv": "devenv",
        "nixpkgs": "nixpkgs",
        "pre-commit-hooks": "pre-commit-hooks"
      }
    }
  },
  "root": "root",
  "version": 7
}
bobvanderlinden commented 1 year ago

I think this is because LD_LIBRARY_PATH is set by devenv. This can lead to packages using a different .so than what they were built with. If the glibc version that the application and one of its libraries are linking to are differring you'll run into trouble.

Always setting LD_LIBRARY_PATH to the $DEVENV_PROFILE/lib might not be a good idea. It should only be set in very specific circumstances (even then it's a less than ideal solution).

@tfmoraes or @jocelynthode could you try:

$ devenv shell
(devenv) $ devenv info
# Expect failure
(devenv) $ unset LD_LIBRARY_PATH
(devenv) $ devenv info
# Should be good?

If that works, should we remove https://github.com/cachix/devenv/blob/971ca55b91f3f326b7067ec92345e4527fc01805/src/modules/mkNakedShell.nix#L102

This might also be related to https://github.com/cachix/devenv/pull/507, as mkShell doesn't set LD_LIBRARY_PATH.

tfmoraes commented 1 year ago

It worked @bobvanderlinden:

❯ devenv info
/nix/store/b38iyffa5xsd26id1400b404c29p3s7d-nix-2.12.0pre20230216_7c91803/bin/nix: /nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224/lib/libc.so.6: version `GLIBC_2.36' not found (required by /nix/store/mm1zi7zaawqi6rahnf6ssarlq0dc9wln-devenv-profile/lib/libstdc++.so.6)
/nix/store/b38iyffa5xsd26id1400b404c29p3s7d-nix-2.12.0pre20230216_7c91803/bin/nix: /nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224/lib/libc.so.6: version `GLIBC_2.36' not found (required by /nix/store/mm1zi7zaawqi6rahnf6ssarlq0dc9wln-devenv-profile/lib/libcurl.so.4)
/nix/store/b38iyffa5xsd26id1400b404c29p3s7d-nix-2.12.0pre20230216_7c91803/bin/nix: /nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224/lib/libc.so.6: version `GLIBC_ABI_DT_RELR' not found (required by /nix/store/1nyg1fvhpz8bx3vn3r9f18zhra2rpbx9-glibc-2.37-8/lib/libresolv.so.2)
❯ set -e LD_LIBRARY_PATH # because I'm on fish shell
❯ devenv info
Inputs:
├───devenv: github:cachix/devenv/971ca55b91f3f326b7067ec92345e4527fc01805?dir=src%2fmodules
├───nixpkgs: github:NixOS/nixpkgs/d6b863fd9b7bb962e6f9fdf292419a775e772891
└───pre-commit-hooks: github:cachix/pre-commit-hooks.nix/fb58866e20af98779017134319b5663b8215d912
    ├───flake-compat: github:edolstra/flake-compat/35bb57c0c8d8b62bbfd284272c928ceb64ddbde9
    ├───flake-utils: github:numtide/flake-utils/5aed5285a952e0b949eb3ba02c12fa4fcfef535f
    ├───gitignore: github:hercules-ci/gitignore.nix/a20de23b925fd8264fd7fad6454652e142fd7f73
    │   └───nixpkgs follows input 'pre-commit-hooks/nixpkgs'
    ├───nixpkgs follows input 'nixpkgs'
    └───nixpkgs-stable: github:NixOS/nixpkgs/9b8e5abb18324c7fe9f07cb100c3cd4a29cda8b8

# env
- DEVENV_DOTFILE: /home/thiago/tmp/mandible_segmentation/.devenv
- DEVENV_PROFILE: /nix/store/mm1zi7zaawqi6rahnf6ssarlq0dc9wln-devenv-profile
- DEVENV_ROOT: /home/thiago/tmp/mandible_segmentation
- DEVENV_STATE: /home/thiago/tmp/mandible_segmentation/.devenv/state
- GREET: devenv
- PYTHONPATH: $DEVENV_PROFILE/lib/python3.11/site-packages

# packages
- python3-3.11.3
...

But there is a problem here. If LD_LIBRARY_PATH is not set how python libs like numpy and torch will work?

bobvanderlinden commented 1 year ago

Nice 👍 LD_LIBRARY_PATH can still be set, but you'll have to be explicit about which package to use. Using all packages for it will quickly become problematic.

See also https://github.com/cachix/devenv/pull/507

That said, even if you explicitly define LD_LIBRARY_PATH, it can still cause trouble.

The underlying problem is that poetry (or other package managers) download prebuilt binaries. These are not always suited for your system (built against a different glibc version for instance). Building them from source is safer, but requires more buildtools to be available in the shell environment.

I'm experimenting with integrating poetry2nix into devenv. It has the right approach and builds everything from source. It however often requires more fiddling than with just poetry.

jocelynthode commented 1 year ago

I think this is because LD_LIBRARY_PATH is set by devenv. This can lead to packages using a different .so than what they were built with. If the glibc version that the application and one of its libraries are linking to are differring you'll run into trouble.

Always setting LD_LIBRARY_PATH to the $DEVENV_PROFILE/lib might not be a good idea. It should only be set in very specific circumstances (even then it's a less than ideal solution).

@tfmoraes or @jocelynthode could you try:

$ devenv shell
(devenv) $ devenv info
# Expect failure
(devenv) $ unset LD_LIBRARY_PATH
(devenv) $ devenv info
# Should be good?

If that works, should we remove

https://github.com/cachix/devenv/blob/971ca55b91f3f326b7067ec92345e4527fc01805/src/modules/mkNakedShell.nix#L102

This might also be related to #507, as mkShell doesn't set LD_LIBRARY_PATH.

I can also confirm that by unsetting LD_LIBRARY_PATH it fixes the issue on my end.

jocelynthode commented 1 year ago

Any idea on how we could fix this issue in a less hackish way?

bobvanderlinden commented 1 year ago

https://github.com/cachix/devenv/pull/507 will be helpful for that.

Alternatively, removing https://github.com/cachix/devenv/blob/c388b8c57116a71174d26b09c0c38b4b6b5bac3a/src/modules/mkNakedShell.nix#L103 could also be considered.

Both solutions will cause problems for people relying on LD_LIBRARY_PATH being set unfortunately.

domenkozar commented 11 months ago

Similar issue happens if you add pkgs.stdenv.cc.cc.lib as a package.

tfmoraes commented 11 months ago

Wrap devenv binary inside a script that removes $DEVENV_PROFILE/lib from LD_LIBRARY_PATH before call devenv is not a fix to this problem?

bobvanderlinden commented 11 months ago

I think you'll still run into issues where other applications will still break. For instance if you use git from your system. You want your system binaries to use system libraries. Otherwise you'll likely run into issues like loading a different (major) glibc version and it being incompatible with the glibc version of system libraries that git also depends on.

LD_LIBRARY_PATH makes things very complex if you allow other tools from your system to be used inside devenv shell next to tools from devenv/nixpkgs.

For python I think the most stable option is to use poetry2nix. There, every python package will be built inside Nix instead of using a prebuilt binary. Packages with native dependencies will directly link to the Nix store path that way. It won't interfere with system-wide packages/binaries/libraries.

tfmoraes commented 11 months ago

I was thinking other way to fix this:

  1. Use autopatchelf after the creation of the virtualenv.
  2. Use something like nix-ld. The main problem is the need of creating wraps.
  3. Use buildFHSUserEnv with packages listed in packages. I think this is the simplest solution, but I don't know if there is some drawbacks.
penguincoder commented 9 months ago

I have seen this happen on my machine (Fedora 38) when I tried to run a newer devenv from inside of an older devenv shell. I just confirmed (with the same version of devenv) that it is blindly adding the paths to LD_LIBRARY_PATH. If this is the issue, it must be adding an incompatible directory with an older glibc.

cargo and rustup solve this in a kind of elegant way in ~/.cargo/env:

#!/bin/sh
# rustup shell setup
# affix colons on either side of $PATH to simplify matching
case ":${PATH}:" in
    *:"$HOME/.cargo/bin":*)
        ;;
    *)
        # Prepending path in case a system-installed rustc needs to be overridden
        export PATH="$HOME/.cargo/bin:$PATH"
        ;;
esac

So we could maybe remove any other /nix/store paths from LD_LIBRARY_PATH and only change it if it does not already contain the expected directory.

domenkozar commented 9 months ago

https://twitter.com/domenkozar/status/1689213491628969985

As part of [http://devenv.sh](https://t.co/mvAnLW2bGd) rewrite, I'm examining all the challenges we face supporting Python, and finding that discovering dynamic libraries is a major pain point.

Libraries like cffi+cairo rely on invoking gcc/ld_config to discover .so files at runtime, which doesn't work with Nix at all.

The typical solution is to set LD_LIBRARY_PATH, which often leads to various issues, such as glibc incompatibilities, etc.

Ideally, Python would use pkg-config to determine where the libraries are installed if it wants to load them at runtime.

Packaging tools like poetry could include a section in pyproject.toml that describes required libraries during configuration and offers a runtime querying API to access the packaging metadata.

Currently our best bet is to provide manylinux wheel for python and make it extendable.

bobvanderlinden commented 9 months ago

From what I gathered, it is possible to force pip/poetry to build the bindings from source. When doing so we do not need to rely on finding .so at runtime, but we do need to make sure to rebuild all packages when something relevant in the environment changes.

Not totally sure though. Poetry2nix does sound like the most stable solution, but it needs quite a bit of UX work to actually use it.

tfmoraes commented 8 months ago

Maybe this can be used https://github.com/GuillaumeDesforges/fix-python

mcdonc commented 4 months ago

I think this will be fixed when:

  1. 920 is fixed.

  2. We fix #921 and then suggest to folks that they use this instead of a "follows"-less nixpkgs-python

        nixpkgs-python:
      url: github:cachix/nixpkgs-python
      inputs:
        nixpkgs:
          follows: nixpkgs

  3. After those issues are fixed, we merge the python-rewrite branch.

domenkozar commented 1 month ago

This is finally fixed in 1.0, great work everyone :)