nix-community / home-manager

Manage a user environment using Nix [maintainer=@rycee]
https://nix-community.github.io/home-manager/
MIT License
6.68k stars 1.75k forks source link

nixos module imports separate <nixpkgs> #616

Closed arcnmx closed 2 years ago

arcnmx commented 5 years ago

The nixpkgs pulled in by home-manager's nixos module may not have access to proper system config options, overlays, or run into weird mismatches when building remotely or with nixops, etc.

Workarounds are easy enough:

{ pkgs, config, ... }:
{
  nixpkgs = { ... };

  home-manager.users.NAME = _: {
    # potentially incomplete when working with obscure configs?
    nixpkgs = { inherit (config.nixpkgs) config overlays system; };

    # this seems to work too
    _module.args.pkgs = pkgs.lib.mkForce pkgs;
  };
}

But it seems like the module should be doing something like this by default?

rycee commented 5 years ago

This is a pretty delicate matter.

The first alternative seems to me the nicest but does indeed seem a bit brittle. If, for example, you want to use nixpkgs-18.09 for the system but nixpkgs-unstable for your user then the system nixpkgs configuration may not be compatible with nixpkgs-unstable.

The second alternative is very straight forward but will prevent configuring the package set using the Home Manager nixpkgs option.

It is not clear to me how to best solve this. Neither of these two options seem solid enough to me to act as a default. I'll have to think about it 🙂

arcnmx commented 5 years ago

If, for example, you want to use nixpkgs-18.09 for the system but nixpkgs-unstable for your user then the system nixpkgs configuration may not be compatible with nixpkgs-unstable.

If you're providing a different nixpkgs for each of nixos system config and the user, you would really want to assign to pkgs inside the home-manager submodule to do this - in which case the nixpkgs config arguments would be ignored anyway. (also home-manager should probably expose a nixpkgs.pkgs option like how nixos does to support this in an obvious and documented way)

Though not quite the same thing, the notes on the nixpkgs nixos function include opinions on a related example:

Unlike in plain NixOS, the nixpkgs.config and nixpkgs.system options will be ignored by default. Instead, nixpkgs.pkgs will have the default value of pkgs as it was constructed right after invoking the nixpkgs function (e.g. the value of import { overlays = [./my-overlay.nix]; } but not the value of (import {} // { extra = ...; }).

If you do want to use the config.nixpkgs options, you are probably better off by calling nixos/lib/eval-config.nix directly, even though it is possible to set config.nixpkgs.pkgs.

For more information about writing NixOS modules, see https://nixos.org/nixos/manual/index.html#sec-writing-modules

Note that you will need to have called Nixpkgs with the system parameter set to the right value for your deployment target.

fritz-k commented 5 years ago

(also home-manager should probably expose a nixpkgs.pkgs option like how nixos does to support this in an obvious and documented way)

That'd be great. I'm currently struggling at using unstable packages in my nixos-managed home.nix. Setting

let
  pkgs = import <nixos-unstable> { config = config.nixpkgs.config; };
in
  ...

in didn't work - due to nixpkgs.nix calling import <nixpkgs> I assume.

flokli commented 5 years ago

Same here - I'd like to be able to build my whole system closure, using home-manager as a NixOS module, without having another <nixpkgs> import happening.

fritz-k commented 5 years ago

I settled on installing all packages from stable and specifying the ones from unstable with

let
  unstable = import <nixos-unstable> { config = config.nixpkgs.config; };
  ...
in {
  nixpkgs.overlays = [
    (pkgself: pkgsuper: {
      somePackage = unstable.somePackage;
      otherPackage = unstable.otherPackage;
    } )
  ];
  ...
}

@flokli That probably isn't the solution to your problem though.

flokli commented 4 years ago

@fritz-k I went with the

nixpkgs = { inherit (config.nixpkgs) config overlays system; };

proposed by @arcnmx now, and it seems to work well :-)

Maybe we should a line about this to the manual.

lovesegfault commented 4 years ago

I understand @arcnmx made some PRs (#993, #992) that were supposed to make it possible to evaluate home-manager without NIX_PATH being set. This, however, does not currently work.

When I try to build my config with NIX_PATH unset I get the following:

❯ NIX_PATH= nix-build -A home.foucault --show-trace
error: while evaluating 'homePkg' at /home/bemeurer/src/nix-config/home/default.nix:3:22, called from /home/bemeurer/src/nix-config/home/default.nix:16:14:
while evaluating the attribute 'activationPackage' at /nix/store/pk81mh4djhpzbf5jqbgnfspn1517zckk-source/home-manager/home-manager.nix:86:18:
while evaluating the attribute 'activationPackage' at /nix/store/pk81mh4djhpzbf5jqbgnfspn1517zckk-source/modules/default.nix:52:3:
while evaluating 'showWarnings' at /nix/store/pk81mh4djhpzbf5jqbgnfspn1517zckk-source/modules/default.nix:16:18, called from /nix/store/pk81mh4djhpzbf5jqbgnfspn1517zckk-source/modules/default.nix:37:12:
while evaluating 'foldr' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/lists.nix:52:20, called from /nix/store/pk81mh4djhpzbf5jqbgnfspn1517zckk-source/modules/default.nix:20:7:
while evaluating 'fold'' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/lists.nix:55:15, called from /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/lists.nix:59:8:
while evaluating 'collectFailed' at /nix/store/pk81mh4djhpzbf5jqbgnfspn1517zckk-source/modules/default.nix:13:19, called from /nix/store/pk81mh4djhpzbf5jqbgnfspn1517zckk-source/modules/default.nix:39:16:
while evaluating the attribute 'assertions' at undefined position:
while evaluating anonymous function at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:84:45, called from undefined position:
while evaluating the attribute 'value' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:379:9:
while evaluating the option `assertions':
while evaluating the attribute 'mergedValue' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:411:5:
while evaluating the attribute 'values' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:405:9:
while evaluating the attribute 'values' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:505:7:
while evaluating anonymous function at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:391:28, called from /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:391:17:
while evaluating 'dischargeProperties' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:464:25, called from /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:392:62:
while evaluating 'dischargeProperties' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:464:25, called from /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:470:11:
while evaluating the attribute 'condition' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:549:14:
while evaluating the attribute 'condition' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:549:14:
while evaluating the attribute 'plugins' at undefined position:
while evaluating anonymous function at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:84:45, called from undefined position:
while evaluating the attribute 'value' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:379:9:
while evaluating the option `programs.tmux.plugins':
while evaluating the attribute 'mergedValue' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:411:5:
while evaluating anonymous function at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:413:22, called from /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:413:9:
while evaluating 'merge' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/types.nix:262:20, called from /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:416:12:
while evaluating anonymous function at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/types.nix:263:35, called from undefined position:
while evaluating anonymous function at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/lists.nix:116:29, called from undefined position:
while evaluating anonymous function at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/types.nix:265:23, called from /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/lists.nix:116:32:
while evaluating the attribute 'optionalValue' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:424:5:
while evaluating the attribute 'values' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:405:9:
while evaluating the attribute 'values' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:505:7:
while evaluating anonymous function at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:391:28, called from /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:391:17:
while evaluating 'dischargeProperties' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:464:25, called from /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:392:62:
while evaluating the attribute 'value' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/types.nix:269:40:
while evaluating the module argument `pkgs' in "/home/bemeurer/src/nix-config/home/modules/tmux.nix":
while evaluating the attribute '_module.args.pkgs' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/attrsets.nix:344:7:
while evaluating anonymous function at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/types.nix:315:29, called from /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/attrsets.nix:344:15:
while evaluating the attribute 'optionalValue.value' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:424:5:
while evaluating the attribute 'values' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:405:9:
while evaluating anonymous function at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:401:19, called from /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:401:14:
while evaluating the attribute 'value._type' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:503:73:
while evaluating the attribute 'value.content' at /nix/store/jvs7dbbyimlq86qfz448kvgmi0nnxhkr-source/lib/modules.nix:564:14:
file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I), at /nix/store/pk81mh4djhpzbf5jqbgnfspn1517zckk-source/modules/misc/nixpkgs.nix:52:18

This, in turn, is due to this unfortunate line in the nixpkgs module of home-manager https://github.com/rycee/home-manager/blob/faa2945606213619dac773b9264c29fa748ad935/modules/misc/nixpkgs.nix#L52-L54

I've tried some of the above tricks, but have been unable to get nix to not evaluate _pkgs. How can I force-out that <nixpkgs>?

Some salient points of my config:

https://github.com/lovesegfault/nix-config/blob/4b7db7b89651c88aff82578257fe06448013c71b/default.nix#L3

https://github.com/lovesegfault/nix-config/blob/4b7db7b89651c88aff82578257fe06448013c71b/home/default.nix#L3-L10

erikarvstedt commented 4 years ago

https://github.com/rycee/home-manager/pull/1059 solves this issue for users the NixOS module.

hpfr commented 4 years ago

I'm using useGlobalPkgs from #1059 and the NixOS module and still having issues with the config namespace. If I try to reference config.services.xserver.dpi (a NixOS module option) within the home-manager configuration, I get error: attribute 'xserver' missing. Any suggestions on this?

misuzu commented 4 years ago

I'm using useGlobalPkgs from #1059 and the NixOS module and still having issues with the config namespace. If I try to reference config.services.xserver.dpi (a NixOS module option) within the home-manager configuration, I get error: attribute 'xserver' missing. Any suggestions on this?

You could do something like this:

dotfiles.nix:

{ pkgs, lib, ... }:
{
  # home-manager config here
}

configuration.nix:

{ config, pkgs, lib, ... }:
let
  dotfiles = with lib; mkMerge [
    (import ./dotfiles.nix)
    {
      someOption = config.services.xserver.dpi;
    }
  ];
in
{
  imports = [
    <home-manager/nixos>
  ];
  home-manager = {
    users = {
      user = dotfiles;
    };
    useGlobalPkgs = true;
    useUserPackages = true;
  };
}
hpfr commented 4 years ago

You could do something like this:

Thanks. I could, but I was using that option as an example. I'd hope to be able to reference arbitrary NixOS config options across files, so I don't know if a localized let expression renaming options would be the best tool.

I was under the impression that erikarvstedt's PR implemented this, given that he said it solves this issue and the original issue stated

The nixpkgs pulled in by home-manager's nixos module may not have access to proper system config options

but if someone can confirm this particular use case doesn't work yet, I'd appreciate it. Sorry if I'm misunderstanding anything

misuzu commented 4 years ago

Thanks. I could, but I was using that option as an example. I'd hope to be able to reference arbitrary NixOS config options across files, so I don't know if a localized let expression renaming options would be the best tool.

I don't really understand what are you trying to do. Could you provide your config? Maybe you would like this better which is basically the same as code above:

{ config, pkgs, lib, ... }:
{
  imports = [
    <home-manager/nixos>
  ];
  home-manager = {
    users = {
      # note: no config here, because it's not system's config but H-M's
      user = { pkgs, lib, ... }:
      {
        someOption = config.services.xserver.dpi;
      };
    };
  };
}

I was under the impression that erikarvstedt's PR implemented this

Isn't it only for options under nixpkgs namespace?

hpfr commented 4 years ago

Isn't it only for options under nixpkgs namespace?

Yeah, I actually took the time to look at the PR now, sorry. You're right. In that case, this issue should probably remain open since there's probably still room for improvement with respect to a way to reference global config.

I still don't fully understand your example, sorry. When I remove config from my user arguments, I can no longer reference home-manager options, such as config.services.polybar.config."bar/main".height, which I use to dynamically configure services.dunst.settings.global.geometry here. I guess what I'm hoping to do is reference NixOS config within home-manager via something like nixos-config.services.xserver.dpi where nixos-config references the systemwide config attribute set so that I can reference all NixOS config options within home-manager without worrying about them colliding with the home-manager config namespace. Is this possible? Again, sorry for my lack of understanding of how NixOS modules interact.

rycee commented 4 years ago

@hpfr Just bind the NixOS config to a different name.

{ config, ... }:
{
home-manager.users =
  let nixosConfig = config;
  in {
    yourusername = { config, ... }: {
      home.file.sysdpi.text = toString nixosConfig.services.xserver.dpi;
    };
  };
}
hpfr commented 4 years ago

Just bind the NixOS config to a different name.

@rycee Thanks, this works, but only locally. I've been defining some of my own modules that I import within my user config, and referencing nixos-config from those module files results in error: undefined variable 'nixos-config' at [where I reference nixos-config in my module]. Do I have to expose options in my home-manager modules every time I want to reference my NixOS config?

hpfr commented 4 years ago

Ok, thanks to rycee's help on IRC, I ended up with this:

configuration.nix

{ config, lib, pkgs, ... }:
{
  home-manager.users.user = let nixos-config = config;
  in { config, lib, pkgs, ... }: {
    _module.args.nixos-config = nixos-config;

    imports = [ ./path/to/my/home-manager/module.nix ];

    some.home-manager.option = nixos-config.some.value;
  };
}

./path/to/my/home-manager/module.nix

{ nixos-config, config, lib, pkgs, ... }:

with lib;

let cfg = config.profiles.polybar;
in {
  options.profiles.polybar.enable = mkEnableOption "my polybar configuration";

  config = mkIf cfg.enable {
    services.polybar = {
      enable = true;
      config."bar/main".height = if nixos-config.services.xserver.dpi == 192 then 34 else 24;
    };
  };
}

I think leveraging Nix to dynamically define configuration is a great use of home-manager and sets it apart from other dotfile management solutions. Maybe it's worth adding this to the manual? In any case, thanks for the help everyone!

stale[bot] commented 3 years ago

Thank you for your contribution! I marked this issue as stale due to inactivity. If this remains inactive for another 7 days, I will close this issue. Please read the relevant sections below before commenting.

If you are the original author of the issue

* If this is resolved, please consider closing it so that the maintainers know not to focus on this. * If this might still be an issue, but you are not interested in promoting its resolution, please consider closing it while encouraging others to take over and reopen an issue if they care enough. * If you know how to solve the issue, please consider submitting a Pull Request that addresses this issue.

If you are not the original author of the issue

* If you are also experiencing this issue, please add details of your situation to help with the debugging process. * If you know how to solve the issue, please consider submitting a Pull Request that addresses this issue.

Memorandum on closing issues

If you have nothing of substance to add, please refrain from commenting and allow the bot close the issue. Also, don't be afraid to manually close an issue, even if it holds valuable information.

Closed issues stay in the system for people to search, read, cross-reference, or even reopen--nothing is lost! Closing obsolete issues is an important way to help maintainers focus their time and effort.

arcnmx commented 3 years ago

So I believe following nixpkgs' example and creating an equivalent config.nixpkgs.pkgs option is probably the most reasonable thing to do here?

flokli commented 3 years ago

Apparently there is now home-manager.useGlobalPkgs = true;, and it's documented in doc/installation.adoc.

I recently stumbled over it, and didn't notice when it got introduced. I assume the same applies to many interacting with this issue.

Can we expose this even more prominently?

austinbutler commented 3 years ago

Apparently there is now home-manager.useGlobalPkgs = true;, and it's documented in doc/installation.adoc.

I recently stumbled over it, and didn't notice when it got introduced. I assume the same applies to many interacting with this issue.

Can we expose this even more prominently?

This had me tripped up for a while today, was wondering why my overlay wasn't working. Setting useGlobalPkgs made it work as I had expected it to.

stale[bot] commented 2 years ago

Thank you for your contribution! I marked this issue as stale due to inactivity. If this remains inactive for another 7 days, I will close this issue. Please read the relevant sections below before commenting.

If you are the original author of the issue

* If this is resolved, please consider closing it so that the maintainers know not to focus on this. * If this might still be an issue, but you are not interested in promoting its resolution, please consider closing it while encouraging others to take over and reopen an issue if they care enough. * If you know how to solve the issue, please consider submitting a Pull Request that addresses this issue.

If you are not the original author of the issue

* If you are also experiencing this issue, please add details of your situation to help with the debugging process. * If you know how to solve the issue, please consider submitting a Pull Request that addresses this issue.

Memorandum on closing issues

If you have nothing of substance to add, please refrain from commenting and allow the bot close the issue. Also, don't be afraid to manually close an issue, even if it holds valuable information.

Closed issues stay in the system for people to search, read, cross-reference, or even reopen--nothing is lost! Closing obsolete issues is an important way to help maintainers focus their time and effort.

flokli commented 2 years ago

Still relevant.

LunNova commented 2 years ago

https://github.com/nix-community/home-manager/pull/2009#issuecomment-997142394

There doesn't seem to be an equivalent approach when using home-manager.lib.homeManagerConfiguration.

ShamrockLee commented 2 years ago

@LunNova IIUC, the flake-based Home Manager configuration always use the nixpkgs from the home manager flake input, and one can adjust it by

{
  inputs.nixpkgs.url = "blablabla";
  inputs.home-manager.inputs.nixpkgs.follows = "nixpkgs";
}
LunNova commented 2 years ago

@ShamrockLee While that works, you get an uncustomized nixpkgs. If you've already imported it, configured it, and added overlays elsewhere in your flake it's wasteful for home-manager to import it again and not very helpful.

I currently use this:

https://github.com/LunNova/nixos-configs/blob/1eda3c70d38a492b98a5fb1989f1c41761edffef/flake.nix#L204-L205

berbiche commented 2 years ago

https://github.com/nix-community/home-manager/blob/63dccc4e60422c1db2c3929b2fd1541f36b7e664/flake.nix#L44-L47 You can override the pkgs argument actually

berbiche commented 2 years ago

Anyway, @LunNova if you could move the discussion to a new issue it would be appreciated given the two problems are different.

The original issue is about using the global nixpkgs configuration when using Home Manager as module.

The original issue was resolved when the home-manager.useGlobalPkgs option was introduced.