bug: Duplicate entries when using builtins.readFile

[supermarin]

Is there an existing issue for this?

• [X] I have searched the existing issues

Issue description

I've just noticed that all the files sourced with builtins.readFile end up having duplicated contents, like it was sourced twice. Tested with .ssh/config, neovim config, .sqliterc, etc. The issue persists across any dotfile that's installed like this:

  home.file.".sqliterc".text = builtins.readFile ./sqliterc;

The resulting file

.headers on
.mode column

.headers on
.mode column

Here's contents of the source, result file and proof that ~/.sqliterc is indeed installed by home-manager:

$ cat dotfiles/sqliterc
.headers on
.mode column
$ cat ~/.sqliterc
.headers on
.mode column

.headers on
.mode column
$ ls -la ~/.sqliterc
lrwxr-xr-x 1 supermarin staff 72 Jun 28 11:25 /Users/supermarin/.sqliterc -> /nix/store/1g5zzz7fcai00dc068w0q75d87mraxig-home-manager-files/.sqliterc

Maintainer CC

No response

System information

Flake based install.
 - system: `0`
 - host os: `Darwin 21.4.0, macOS 12.3.1`
 - multi-user?: `no`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.9.1`
 - channels(root): `"nixpkgs"`
 - channels(supermarin): `""`

...
From flake.lock

    "home-manager": {
      "inputs": {
        "nixpkgs": [
          "nixpkgs"
        ],
        "utils": "utils"
      },
      "locked": {
        "lastModified": 1656367977,
        "narHash": "sha256-0hV17V9Up9pnAtPJ+787FhrsPnawxoTPA/VxgjRMrjc=",
        "owner": "nix-community",
        "repo": "home-manager",
        "rev": "3bf16c0fd141c28312be52945d1543f9ce557bb1",
        "type": "github"
      },
      "original": {
        "owner": "nix-community",
        "repo": "home-manager",
        "type": "github"
      }
    },

Tested on nixOS (home-manager also used as a module from configuration.nix vs darwin.nix) and the issue is not there, so likely related to nix-darwin. Related closed issue on home-manager: https://github.com/nix-community/home-manager/issues/3059

[supermarin]

I've finally managed to narrow this down. Interested in maintainers opinion on how best to proceed, as I believe the current state is a bit confusing. I think there should be a check added either in home-manager or nix-darwin. Will try to repro on nixos, and report if it's darwin only.

The problem

A few months ago I've switched to flakes. I'm using home-manager as a module both in nixos and darwin, meaning that you don't have a separate home-manager command. Everything is done via nixos-rebuild or darwin-rebuild.

As a part of switching to flakes, somehow I've ended up with a double home-manager.users.USER definitions, one in flake.nix (as suggested by home-manager manual), and one in darwin-configuration.nix that has been there since before.

#flake.nix
{
  inputs = {
    nixpkgs.url = github:nixos/nixpkgs;
    home-manager.url = github:nix-community/home-manager;
    home-manager.inputs.nixpkgs.follows = "nixpkgs";
    darwin.url = github:lnl7/nix-darwin;
    darwin.inputs.nixpkgs.follows = "nixpkgs";
  };

  outputs = { self, nixpkgs, home-manager, darwin }:
  {
    darwinConfigurations = {
      komputer = darwin.lib.darwinSystem {
        system = "aarch64-darwin";
        modules = [ 
          ./darwin-configuration.nix
          home-manager.darwinModules.home-manager {
            home-manager.useGlobalPkgs = true;
            #home-manager.useUserPackages = true;
            home-manager.users.supermarin = import ../home.nix;
          }
        ];
      };
    };
  };
}

# darwin-configuration.nix
{
  home-manager.users.supermarin = {
    imports = [ ../home.nix ];
  };
}

The fix

I'm guessing there should be a test on home-manager activation that should catch this. I was expecting an error from nix (duplicate entries), but it seems like the inner-referenced home-manager resolved to something else compared to the one in flake.nix that was passed as input, even though I've assigned it's nixpkgs.follows to nixpkgs.

[malob]

I agree to the extent there is something to fix here, it would be on home-manager's end rather than nix-darwin's end, though I'm not sure there is in fact something to fix.

The home-manager.users option is of type attrsOf hmModule (where hmModule is a submodule), and I'm all but certain the duplication is the result of the default behavior of merging options of type submodule. So by defining home-manager.users.supermarin twice, you are defining two sets of options for that submodule which are merged. So in your case, this results in the home.file.".sqliterc".text option being defined twice, and since the merging behavior for that option (which has type types.nullOr types.lines) is to concatenate the two values separated by a newline, the resulting file contains duplicate content. (I expect you know most of this, but figured I'd go into details anyways, in case others with similar issues stumble upon this issue.)

[supermarin]

@malob thanks for the great explanation in detail. I agree with you and will re-open the issue on HM side. Don't have any great ideas right off the top of my head because of the nature of this issue.