hercules-ci / gitignore.nix

Nix functions for filtering local git sources
The Unlicense
242 stars 19 forks source link

Wrapping inner filter in gitignoreSource is not the same as wrapping other way around? #30

Open robinp opened 4 years ago

robinp commented 4 years ago

Test: cd subdir, nix-build in clean state, then touch src/apple.o, where *.o is in the .gitignore in the project root directory.

This doesn't work (rebuild gets triggered):

src = gitignore.gitignoreSource (pkgs.lib.sources.sourceByRegex ./.
              [ "package.*json"
                "src.*"
                "scss.*"
                "tests.*"
                "webpack\\.config\\.js"
                "jest.*\\.config\\.js"
              ];
      });

This works (no rebuild):

 src = pkgs.lib.sources.sourceByRegex (gitignore.gitignoreSource ./.)
              [ "package.*json"
                "src.*"
                "scss.*"
                "tests.*"
                "webpack\\.config\\.js"
                "jest.*\\.config\\.js"
              ];
      });
3noch commented 4 years ago

We ran into a similar issue that had baffling effects. When we used gitignoreSource (pkgs.lib.cleanSource src), ghcid would take up to 60 seconds to notice changes to our files and reload them; when we switched the order of those around, everything worked as expected!

https://github.com/obsidiansystems/obelisk/pull/666

EDIT: We diffed the output store paths from both and as far as we could tell, everything was identical. What seemed to be the real culprit is that ghc itself would only use one core when gitignoreSource was applied second, but it would use all cores the other way. Clearly there is some dark magic causing this interaction, but we weren't able to discover what it was.

roberth commented 4 years ago

@3noch, composing cleanSourceWith-derived functions is like composing the boolean and operator. Denotationally the composition is symmetric. As you have noticed, the output is the same. But just like and, the operational semantics are not symmetric.

@robinp I've seen hashes differ because of a name that changes. The name logic in cleanSourceWith is a bit complicated. Could you check the hashes and names by calling nix-instantiate on src? Sorry for the late reply.

3noch commented 4 years ago

@roberth Sure, but the particular effects we witnessed are completely inexplicable currently... I'm mostly reporting it here in case someone else ever sees anything remotely related.