Gitignore works differently on different machines

[NorfairKing]

I've run into (another) unreproducible issue because gitignore.nix reads global ignore files: https://github.com/NorfairKing/smos/issues/226#issuecomment-913126182

I've not formed any opinion on this choice yet, but I would like to learn about the tradeoff so that I may make my builds reproducible again. Where can I learn about this?

[roberth]

This choice was made because git supports this:

Patterns which a user wants Git to ignore in all situations (e.g., backup or temporary files generated by the user’s editor of choice) generally go into a file specified by core.excludesFile in the user’s ~/.gitconfig. Its default value is $XDG_CONFIG_HOME/git/ignore. If $XDG_CONFIG_HOME is either not set or empty, $HOME/.config/git/ignore is used instead.

-- man gitignore

The alternative design choice is to ignore the user ignores and require all possible tool files to be added to project .gitignores, leading to similar source repro issues when user-specific rules are missing from projects.

The README mentions:

Reads user git configuration; no need to bother your team with your tool config.

You're right that the choice is a tradeoff, so I'm open to making this behavior configurable here. It's worth a comparison.

[NorfairKing]

Thanks for the info!

I'm open to making this behavior configurable here.

I'd be happy to test it out! Even if it's non-default behaviour.. If you point me to the relevant lines then I could look into implementing a PR for it.

[roberth]

You could make it conditional here https://github.com/hercules-ci/gitignore.nix/blob/211907489e9f198594c0eb0ca9256a1949c9d412/find-files.nix#L109 and add parameters all the way up. I'd like the 'public' interface gitignoreSource and gitignoreFilter to be backcompatible. You could introduce *With functions; e.g.gitignoreSourceWith { enableUserRules = false; } ./.

[NorfairKing]

@roberth I have a fork here now, but I haven't had the chance to try it out:

https://github.com/NorfairKing/gitignore.nix/blob/a0b15befbd3e5ae9b8b89d2895bcf3ed964f8814/find-files.nix#L2

[voidus]

My 2 cents as someone who just wasted a few hours on this:

I think the difference is how files which are both tracked by git and listed in a gitignore are handled. Once added (possibly using git add -f ignored_file) git doesn't care about ignore files anymore. This way, If I'm checking out a repository which includes files that are in my personal gitignore, I don't have issues.

So, without having looked into this project a whole deal, I'd humbly suggest that files which are tracked by git should always be considered, regardless of them being matched by any ignore files.

When I'm running a command like

nix-build -E 'import (builtins.fetchGit {
      url = "https://github.com/norfairking/smos";
      rev = "82707699b446bc431f5233e5dc18c7ec3dd679fd";
      ref = "release";
    })'

I am most definitely not expecting my local git configuration to result in missing files during the checkout.

I thought the package was broken and reported https://github.com/NorfairKing/smos/issues/237. The error was very hard to diagnose, I eventually found it by straceing nix-build

[NorfairKing]

@voidus Congrats on the digging!

[roberth]

I've updated the comparison.

I'm becoming increasingly convinced that gitignores are a fight that can't be won. If you really care, you'll want to use explicit inclusion rules, rather than implicit exclusion rules. While explicit inclusion rules themselves can still be unexpected, they don't lead to long debugging sessions, and they are the best solution to needless rebuilds and cache misses caused by accidentally included files.

Explicit inclusion rules can be created with cleanSourceWith and combinators like these.