Generated code through pypi2nix is not usable anymore (pythonPackages change)

[fiksn]

I believe this regression originated from https://github.com/NixOS/nixpkgs/commit/b57c5d44569e236edbc142c0f95772fca2bb6dad. @Ericson2314, can you perhaps help me out here?

I am trying with requirements.nix generated through pypi2nix 2.0.4 on NixOS 21.03.

One thing I get is:

error: anonymous function at /nix/store/84lx5jj7xvlxx4dz245w4hk57d6sfwqg-nixpkgs-21.03pre264143.7d71001b796/nixpkgs/pkgs/top-level/python-packages.nix:9:1 called without required argument 'lib'

but ok I've just added lib here like this:

pythonPackages =
  import "${toString pkgs.path}/pkgs/top-level/python-packages.nix" {
    inherit pkgs;
    inherit (pkgs) stdenv lib;
    python = pkgs.python3;
  };

then I get

error: value is a function while a set was expected

I suppose the reason is that python-packages.nix was refactored and it now returns a function, so this autogenerated requirements.nix should do something different in order to get pythonPackages.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/help-me-understand-scope-so-i-can-use-pypi2nix-requirements-nix-again/11061/1

[fiksn]

So I've solved this by replacing:

pythonPackages =
  import "${toString pkgs.path}/pkgs/top-level/python-packages.nix" {
    inherit pkgs;
    inherit (pkgs) stdenv;
    python = pkgs.python3;
  };

with:

pythonPackages' =
  import "${toString pkgs.path}/pkgs/top-level/python-packages.nix" {
    inherit pkgs;
    inherit (pkgs) stdenv lib;
    python = pkgs.python3;
  };
  pythonPackages = pkgs.lib.makeScope pkgs.newScope pythonPackages';

but I have to admit, that I have no clue what I am doing here :P That's why I've asked about "scope" on NixOS Discourse. It reminds me a bit about fixed-point evaluation and overrides but again this concept of scope which is apparently for cross-compiling and splices is quite alien to me at the moment.

---

Update: here is the thing that works for me in a backward compatible (but probably awkward) way. Quick fix in case anyone else stumbles on this. I know there is callPackage (https://nixos.org/guides/nix-pills/callpackage-design-pattern.html) but that didn't really work well (probably again has to do with scope).

  pythonPackagesImport = import "${toString pkgs.path}/pkgs/top-level/python-packages.nix";
  needsLib = builtins.elem "lib" (pkgs.lib.attrNames (builtins.functionArgs pythonPackagesImport));
  pythonPackages' = pythonPackagesImport ({
    inherit pkgs;
    inherit (pkgs) stdenv;
    python = pkgs.python3;
  } // (if needsLib then { inherit (pkgs) lib; } else {}));
  pythonPackages = if builtins.isFunction pythonPackages' then pkgs.lib.makeScope pkgs.newScope pythonPackages' else pythonPackages';