nix-community / flakelight

Framework for simplifying flake setup [maintainer=@accelbread]
MIT License
229 stars 6 forks source link

`inputs` in `nixosModule` is not available for external flake #22

Open ratson opened 5 months ago

ratson commented 5 months ago
nix build "github:ratson/bug-report/flakelight-nixos-module?dir=flake2#nixosConfigurations.vm2.config.system.build.vm"

gives

error: attribute 'inputs'' missing

While the following is working,

nix build "github:ratson/bug-report/flakelight-nixos-module?dir=flake1#nixosConfigurations.vm1.config.system.build.vm"

Source code in https://github.com/ratson/bug-report/tree/flakelight-nixos-module

@accelbread Would you check if usage in flake2/flake.nix is correct?

FYR, here is a copy of it,

{
  inputs = {
    nixpkgs.url = "nixpkgs/nixos-unstable";
    flake1.url = "path:../flake1";
  };
  outputs = { nixpkgs, flake1, ... }@inputs:
    let
      system = "x86_64-linux";
    in
    {
      nixosConfigurations.vm2 = nixpkgs.lib.nixosSystem {
        inherit system;

        modules = [
          flake1.nixosModules.default
        ];
      };

      packages.${system}.hello2 = flake1.packages.${system}.hello1;
    };
}

There is test.sh script in the repo to run locally to verify the bug.

accelbread commented 4 months ago

Sorry for delay; have been busy.

The inputs' arg is provided by flakelight, so is not available in second flake. The args to modules come from the nixosSystem as well, so inputs' in a module refers to the inputs of the flake defining the system, not the one that defined the module.

If you want to use packages from another flake, I'd recommend having the module use the package from the pkgs arg, and having the second flake's system use the first's overlay.

ratson commented 4 months ago

@accelbread Should the inputs' be provided to flake1.nixosModules.default with flake1's inputs? Or is there a way to refer to flake1's inputs in flale1.nixosModules, so that it can used by another flake?

ratson commented 4 months ago

To better illustrate the problem, I updated flake1 to include vm2,

https://github.com/ratson/bug-report/blob/flakelight-nixos-module/flake1/flake.nix#L13-L19

      nixosConfigurations.vm2 = nixpkgs.lib.nixosSystem {
        inherit system;

        modules = [
          self.nixosModules.default
        ];
      };

It is strange that nix build ./flake1#nixosConfigurations.vm1.config.system.build.vm works, while nix build ./flake1#nixosConfigurations.vm2.config.system.build.vm fails.

Here is the vm1 code,

https://github.com/ratson/bug-report/blob/flakelight-nixos-module/flake1/nix/nixos/vm1.nix

{ inputs, ... }:

{
  system = "x86_64-linux";

  modules = [
    inputs.self.nixosModules.default
  ];
}
accelbread commented 4 months ago

if you change vm2 to not call nixosSystem explicitly, it will work:

      nixosConfigurations.vm2 = {
        inherit system;

        modules = [
          self.nixosModules.default
        ];
      };

If you don't call it, flakelight can propagate inputs and etc. to the nixos invocation since it calls it. If you call it yourself, flakelight can't add its stuff. You can call it yourself and add the flakelight stuff by adding config.propogationModule to your modules. Can see here: https://github.com/nix-community/flakelight/blob/2a96b83bf6ffad4b8a15afe19e883ca1a1720d88/API_GUIDE.md?plain=1#L965-L977

ratson commented 4 months ago

self.nixosModules.default should propagate inputs and moduleArgs for outputs, so non-flakelight users could use it.

I don't know if it can be generically implemented or not, here is a working example,

https://github.com/ratson/bug-report/blob/d0034228e83bb99f30bb60ea905bc2ee262ec364/flake1/flake.nix#L21-L29

accelbread commented 4 months ago

Yeah the problem with that approach is that the parameters are no longer named, and the parameter names affect how modules are passed args. A module with { ... }@args will only get specialArgs, not regular args. Additionally, can't tell if a parameter is meant to be for the NixOS module system or meant to be handled by wrapper.

In order to use host flake module args, you'd have to capture them.

Something like the following works:

nixosModules = { inputs, ... }: {
  default = { pkgs, ... }: {
    environment.systemPackages = [
      inputs.self.packages.${pkgs.system}.hello1
    ];
  };
};

That way inputs in the default module is captured from defining flake rather than NixOS module args.

To autoLoad that would have to use nix/nixosModules.nix. Can't use its own file by default. Or could have a nix/nixosModules.nix that autoloads nix/nixosModules/ but calls it first with moduleArgs allowing:

{ inputs, ... }: { pkgs, ... }: {
  environment.systemPackages = [
    inputs.self.packages.${pkgs.system}.hello1
  ];
};

However, can't do the above automatically since given a function, we don't know if flakelight should call it or not. We could perhaps try calling it and seeing if result is a function, and if so it needs to be called else not. This could cause an evaluation failure though if its not meant to be called, and we passed the wrong args. Theres also no way to catch hard errors. For bundlers we use a similar trick where we wrap the function, capturing moduleArgs, and when called it figures out if the function is nested, and passes args accordingly, but this works since it has the real args. That might potentially work here as well, as long as we can use lib.setFunctionArgs to have the same args on wrapped function. Thought that would mean turning every exported module into a functor, which I'm not sure is a good idea.

Edit: wrapping the function like I mentioned at the end would not work actually, as no way to extract the module named args from the function value without evaluating it, since the real module named args might not be the top level function.

ratson commented 4 months ago

@accelbread Thanks for pointing out the lib.setFunctionArgs function, didn't know that.

I have generalized the wrapper code to be,

https://github.com/ratson/bug-report/blob/f9e2bd01f644756953f10443343e10934cdac367/flake1/flake.nix#L29-L34

      nixosModules.wrapped =
        let
          f = lib.toFunction import ./nix/nixosModules/_default.nix;
          g = args: f (args // { inherit inputs; });
        in
        lib.setFunctionArgs g (lib.functionArgs f);
accelbread commented 4 months ago

You'll also need to filter out inputs, so that if their nixpkgs modules don't have an inputs arg it would still work, like follows:

      nixosModules.wrapped =
        let
          f = lib.toFunction import ./nix/nixosModules/_default.nix;
          g = args: f (args // { inherit inputs; });
        in
        lib.setFunctionArgs g (lib.removeAttrs (lib.functionArgs f) ["inputs"]);

How I'd do modules that need moduleArgs from defining flake is set nix/nixosModules/default.nix to:

{ lib, flakelight, moduleArgs, ... }:
lib.mapAttrs (_: v: v moduleArgs) (flakelight.importDir ./.)

If you have that, then your module files are functions returning modules, for example nix/nixosModules/_default.nix for nixosModules.default could be:

{ inputs, moduleArgs, ... }:
{ pkgs, config, ... }: {
  environment.systemPackages = [ inputs.self.packages.${pkgs.system}.hello1 ];
}
ratson commented 4 months ago

Right now, I am putting all modules requires inputs in flake.nix as suggested,

nixosModules = { inputs, ... }: {
  default = { pkgs, ... }: {
    environment.systemPackages = [
      inputs.self.packages.${pkgs.system}.hello1
    ];
  };
};

@accelbread Do you think it is possible to utilize file extension convention, e.g. ./nixosModules/[name].functor.nix to support the file content like

{ inputs, ... }: { pkgs, ... }: {
 environment.systemPackages = [
   inputs.self.packages.${pkgs.system}.hello1
 ];
}

Or provide such functionality via extra flakelight module?

accelbread commented 4 months ago

Yeah, a folder name could be used. A flakelight module could add a nixosModulesFn option or something, which would automatically work with autoloading.