lib and pkgs.lib loose sync when trying to overlay lib

[ony]

Describe the bug

Attempt to add overlay lib have effect only on pkgs module argument attrset and not on lib.

Steps To Reproduce

Steps to reproduce the behavior:

1. Prepare dummy NixOS configuration

   # flake.nix
   {
   outputs = { nixpkgs, ... }@inputs: with inputs; {
   nixosConfigurations.ony = nixpkgs.lib.nixosSystem rec {
     system = "x86_64-linux";
     modules = [ ./configuration.nix ];
   };
   defaultPackage.x86_64-linux = self.nixosConfigurations.ony.config.system.build.toplevel;
   };
   }

   { lib, pkgs, ... }:
   {
   nixpkgs.overlays = [
   (final: prev: {
     lib = prev.lib // {
       licenses = prev.lib.licenses // {
         sspl = prev.lib.licenses.sspl // {
           shortName = "sspl";  # try to fix upper case SSPL not matching attribute
         };
       };
     };
   })
   ];
   nixpkgs.config = {
   # blocklistedLicenses = with lib.licenses; [ sspl ]; # => ‘SSPL’ is not an attribute of lib.licenses
   blocklistedLicenses = with pkgs.lib.licenses; [ sspl ]; # => ‘sspl’ is not an attribute of lib.licenses
   };
   }

2. Run nix flake check

Expected behavior

• There should be no difference between using lib or pkgs (via pkgs.lib) arguments of module.
• Overriding lib in overlay should have effect on check-meta.nix evaluations too. That message indicates that licenses referenced in blocklistedLicenses and in attrset lib.licenses were different even if sspl attribute was found. See comment for issue with SSPL.

Screenshots

When using lib.licenses (no effect from overlay)

zsh%  nix flake check
error: ‘SSPL’ is not an attribute of lib.licenses
(use '--show-trace' to show detailed location information)

When using pkgs.lib.licenses (license do not match one referenced in check-meta.nix)

zsh%  nix flake check
error: ‘sspl’ is not an attribute of lib.licenses
(use '--show-trace' to show detailed location information)

(notice lower case version of shortName)

Additional context

When there is requirement for blocklistedLicenses to reference only licenses from lib.licenses, we need a way extend that attrset to allow having packages and licenses outside of main nixpkgs repo.

Notify maintainers

Metadata

nix run nixpkgs\#nix-info -- -m

• system: "x86_64-linux"
• host os: Linux 5.10.81, NixOS, 21.11 (Porcupine)
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.4
• channels(root): "nixos-21.11.334684.1158f346391"
• channels(nikolay): "nixpkgs-unstable-21.11pre310022.14b0f20fa1f"
• nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

[roberth]

It is not possible to override lib from within a module, as this would create an infinite recursion:

To evaluate config.system.build.toplevel (ie the "whole" config), we must evaluate all module configs, many of which are defined through lib.mkIf and therefore lib, which would have to come from, again, the configs.

To override the lib module argument, you'll have to pass the modified lib to the nixosSystem function.

cc @ncfavier If we do make nixosSystem use the final lib, we can recommend let lib = nixpkgs.lib.extend foo; in lib.nixosSystem. Working with only one lib like that is perhaps easier.

[roberth]

 rec {

This is redundant. It's generally a good idea not to use it, especially in places where you can access the same values through a "fixed point" like the flake self argument or an overlay's final argument.

[ncfavier]

cc @ncfavier If we do make nixosSystem use the final lib, we can recommend let lib = nixpkgs.lib.extend foo; in lib.nixosSystem. Working with only one lib like that is perhaps easier.

Yes, that's exactly the use case I had in mind (see my config for example)

[ncfavier]

@ony so in your case, something like this should work:

{
  outputs = { nixpkgs, ... }@inputs: with inputs; let
    lib = nixpkgs.lib.extend (final: prev: {
      licenses = prev.licenses // {
        sspl = prev.licenses.sspl // { shortName = "sspl"; };
      };
    });
  in {
    nixosConfigurations.ony = lib.nixosSystem {
      inherit lib; # might become unnecessary in the future
      system = "x86_64-linux";
      modules = [ ({ lib, ... }: ... lib.licenses.sspl ...) ];
    };
    defaultPackage.x86_64-linux = self.nixosConfigurations.ony.config.system.build.toplevel;
  };
}

[ony]

@ncfavier , thank you for lib.extend that saves some boilerplate.
Though using that modified lib.nixosSystem and passing into it modified lib do have effect on module lib, it do not effect pkgs.lib (opposite to nixpkgs.overlays). And it still fails on finding sspl in lib.licenses in check-meta.nix.

Could it be because of this?

https://github.com/NixOS/nixpkgs/blob/c4bb9084aa159f029ae9e35edb580d22f0ed5fdb/pkgs/stdenv/generic/default.nix#L1 https://github.com/NixOS/nixpkgs/blob/c4bb9084aa159f029ae9e35edb580d22f0ed5fdb/pkgs/stdenv/generic/default.nix#L54 https://github.com/NixOS/nixpkgs/blob/c4bb9084aa159f029ae9e35edb580d22f0ed5fdb/pkgs/stdenv/generic/make-derivation.nix#L1-L7

[ncfavier]

I was thinking about that earlier. It seems like a complete solution would look like introducing a concept of an "ambient" lib that's passed around the files in pkgs instead of importing ../lib. Or maybe just adding a simple { inherit lib; } overlay in specific places (although this wouldn't solve the checkMeta situation).

[ony]

I wonder if simple solution would be to not depend on lib.licenses and lib.platforms in check-meta.nix :slightly_smiling_face: .

@roberth , I would expect recursion only in case if overlay overrides something used to lookup and combine overlays. Otherwise it should be able to traverse to original "bootstrap" attrset through all //.

[ncfavier]

Opened https://github.com/NixOS/nixpkgs/pull/157056