How to deal with pep-0668 / externally-managed-environment

[SimonFischer04]

Running npm run npip -- install toposort I get the usual externally-managed-environment error.

First thought:

Python (recently) enforced using Virtual Environments (on most Linux distros at least). Some links:

• https://stackoverflow.com/questions/75608323/how-do-i-solve-error-externally-managed-environment-every-time-i-use-pip-3
• https://peps.python.org/pep-0668/

How should this project dealt with that?

I don't know if it's in scope of the project to create a virtual environment since the readme explicitly states "alternative to a python virtual environment", but what would be the alternative? This package being completely broken on recent python versions?

---

After looking into this a bit more: Isn't that kind of an false-alarm? (As I understand it) this project's whole goal is to create a local python_modules directory and a nopy python wrapper using that directory (so basically re-creating what a venv would do)? Then this should be suppressed somehow?

[Filyus]

From the nopy`s readme:

Behind the scene, nopy uses python's per user site-packages directory mechanism. Specifically, when python or pip are invoked indirectly by way of the nopy or npip wrappers, the PYTHONUSERBASE environment variable is modified to reference the python_modules directory contained in the node.js project, the one alongside package.json. This tells python to look there for installed python packages. Additionally, pip is invoked with the --user option, which causes it to install packages in that python_modules directory.

Possible solutions:

1. Use a venv instead of the user site-packages (probably the most modern solution)
2. Use pip's argument --break-system-packages
3. Set PIP_BREAK_SYSTEM_PACKAGES environment variable
4. Delete or rename /usr/lib/python3.XX/EXTERNALLY-MANAGED
5. Add following lines to ~/.config/pip/pip.conf:

   [global]
   break-system-packages = true

I think there just a line args.push("--break-system-packages"); in src/npip.js needed:

return installed.then(() => {
  if (command == "install" && args.length === 1) {
    let deps = Object.assign({}, json.python.dependencies, json.python.devDependencies);
    let count = 0;
    for (let name in deps) {
      if (Object.prototype.hasOwnProperty.call(deps, name)) {
        let version = deps[name];
        args.push(name + version);
        ++count;
      }
    }

    if (count === 0) {
      console.log("npip has no dependencies listed in package.json to install.");
      return 0;
    }
  }

  if (command === "install" || command === "freeze" || command === "list")
    args.splice(1, 0, "--user");

  // Run pip as a python module so that pip itself does not need to be on PATH, only python.
  args.unshift("-m", "pip");

  return spawnPython(args, {
    package: pkg,
    interop: "status",
    throwNonZeroStatus: false,
    spawn: {
      stdio: "inherit",
    },
  });
});

[Filyus]

I need to say, venv is modern, but it is a some sort of a hack with extending PATH and clearing PYTHONHOME environment variables, probably it won't work at deeper levels with the current implementation because of weird one level _OLD_* environment variables. Also, it uses different folders for Windows and for other systems for python and pip. Yes, it stores python and pip executables, this is not like NodeJS does..