Open danbst opened 5 years ago
Profpatsch
commented
5 years ago If it is inside your project, you can use https://github.com/siers/nix-gitignore (which went into nixpkgs master a month or so ago) or https://github.com/Profpatsch/nixperiments/blob/master/filterSourceGitignore.nix (which is a bit more restrictive but should have less corner cases).
If it is from a transitive dependency lorri picks up, we’ll have to think of something and this is likely a bug.
danbst
commented
5 years ago No, it is the problem with source-less nix-shells
$ cat shell.nix
with import <nixpkgs> {};
mkShell {
buildInputs = [ hello ];
}
$ touch bla
direnv: loading .envrc
direnv: using nix
direnv: export +AR +AS +CC +CONFIG_SHELL +CXX +HOST_PATH +IN_NIX_SHELL +LD +NIX_BINTOOLS +NIX_BINTOOLS_WRAPPER_x86_64_unknown_linux_gnu_TARGET_HOST +NIX_BUILD_CORES +NIX_BUILD_TOP +NIX_CC +NIX_CC_WRAPPER_x86_64_unknown_linux_gnu_TARGET_HOST +NIX_ENFORCE_NO_NATIVE +NIX_HARDENING_ENABLE +NIX_INDENT_MAKE +NIX_LDFLAGS +NIX_LOG_FD +NIX_STORE +NM +OBJCOPY +OBJDUMP +RANLIB +READELF +SIZE +SOURCE_DATE_EPOCH +STRINGS +STRIP +_PATH +buildInputs +builder +configureFlags +depsBuildBuild +depsBuildBuildPropagated +depsBuildTarget +depsBuildTargetPropagated +depsHostHost +depsHostHostPropagated +depsTargetTarget +depsTargetTargetPropagated +doCheck +doInstallCheck +name +nativeBuildInputs +nobuildPhase +origArgs +origBuilder +origPATH +origSystem +out +outputs +phases +propagatedBuildInputs +propagatedNativeBuildInputs +shell +stdenv +strictDeps +system ~PATHIn my case something like touch bla occures quite often to make those envrc reloadings annoying.
EDIT: wait, this works just fine. I have still to extract the problem
infinisil
commented
5 years ago I've just tried lorri with my small Haskell project. It uses nixpkgs' lib.sourceByRegex to filter what gets included:
{
soph = (self.callCabal2nix "soph" (lib.sourceByRegex ./. [
"^\\src.*$"
"^.*\\.cabal$"
"^LICENSE$"
]) {}).overrideAttrs (/** ... **/);
}However this ends up including probably every file in that directory, as seen from the debug output:
infinisil
commented
5 years ago Some more info:
$ nix-instantiate --eval --strict --read-write-mode -A src
{ _isLibCleanSourceWith = true; filter = <LAMBDA>; origSrc = /home/infinisil/prj/soph;
outPath = "/nix/store/9blqrn193vmlvzrv5y612rhkrmn77yrj-soph"; }
$ tree -A /nix/store/9blqrn193vmlvzrv5y612rhkrmn77yrj-soph
/nix/store/9blqrn193vmlvzrv5y612rhkrmn77yrj-soph
├── LICENSE
├── soph.cabal
└── src
├── Config.hs
├── Hashing.hs
├── Log.hs
└── Main.hs
1 directory, 6 files
anka-213
commented
5 years ago I have the same problem as @Infinisil with the added detail that every time the git lockfile .git/index.lock is touched, lorri will rebuild the environment. Since my shell prompt checks the git status at every prompt, this is every time any command is run.
infinisil
commented
5 years ago @anka-213 Yeah I'm having that problem as well :(
dudebout
commented
5 years ago @Infinisil, your problem stems from the first line in your shell.nix file:
with import ./. {};Because it imports "the current directory", that directory ends up being watched recursively by lorri. By changing this line to:
with import ./default.nix {};you can bypass the issue.
dudebout
commented
5 years ago I have noticed two other cases that make lorri watch more directories than one would want:
lorri watches every directory in that checkout, including the .git directory and its subdirectoriessrc that is not computed from filterSource, then lorri watches all the subdirectories of src. I think this is because:
filterSource, the src is added to the nix store via primops:addPath which does not echo anythingfilterSource, the src is added to the nix store via eval:copyPathToStore which echoes "copied source" and leads lorri to watch the directory(Arguably using 1. and 2. are not good practice, but we should document these corner cases for users and in case the logging in nix changes which could cause changes in behavior)
dudebout
commented
5 years ago Created https://github.com/target/lorri/pull/21 to document my findings.
nyarly
commented
5 years ago One case I think I've discovered: I'm working on a Rubygem, and bundix adds an entry for the gemspec as source = ./.
Profpatsch
commented
5 years ago Update: we have implemented the default.nix logic now.
If you can, please try out the https://github.com/target/lorri/pull/146 branch (you can clone, switch to branch and then use lorri self-upgrade local /path/to/checkout to do that).
There are still more issues with watching the minimal amount of files necessary, so please report any inconsistencies that you see.
masaeedu
commented
4 years ago @dudebout The README currently suggests a workaround for 2. Is there a workaround for 1?
I'm using niv to manage my nixpkgs checkout, and lorri seems to be in an infinite loop:
[2019-11-30T05:17:04Z DEBUG lorri::watch] Event path ("/mnt/data/depot/git/haskell/repos/filterable/.git/index.lock") parent ("/mnt/data/depot/git/haskell/repos/filterable/.git") matches watched path
[2019-11-30T05:17:04Z DEBUG lorri::watch] Watch Event: RawEvent {
path: Some(
"/mnt/data/depot/git/haskell/repos/filterable/.git/index.lock",
),
op: Ok(
REMOVE,
),
cookie: None,
}
[2019-11-30T05:17:04Z INFO lorri::watch] identified file removal: "/mnt/data/depot/git/haskell/repos/filterable/.git/index.lock"
[2019-11-30T05:17:04Z DEBUG lorri::watch] Received event: RawEvent {
path: Some(
"/mnt/data/depot/git/haskell/repos/filterable/.git/index.lock",
),
op: Ok(
REMOVE,
),
cookie: None,
}I am not aware of such a workaround. I have a feeling that in most cases this should not be a problem though, since working on nixpkgs or working on the project backed by lorri are fairly distinct operations. If I remember correctly, I noticed 1) while investigating this issue, but this was not something that was in the way.
Are you sure niv is using a git checkout of nixpkgs? From what niv does, and from reading its README, it does not seem to be the case, at least in its default mode of operation.
On Sat, Nov 30, 2019 at 12:19 AM Asad Saeeduddin notifications@github.com wrote:
@dudebout https://github.com/dudebout The README currently suggests a workaround for 2. Is there a workaround for 1?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/target/lorri/issues/6?email_source=notifications&email_token=AAC5LGSKQDW2NYHERXBEW2TQWHZXHA5CNFSM4HCHAFNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFPZW4Y#issuecomment-559913843, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAC5LGTKJRZ6KS4WAIXP6H3QWHZXHANCNFSM4HCHAFNA .
masaeedu
commented
4 years ago @dudebout You're right, it looks like niv uses a tarball. The .git folder that's causing an infinite loop seems to be the actual project I'm working on.
willbush
commented
4 years ago Note regarding the issue where src = ./. causes rebuilding on any file change applies to root when using developPackage.
let
# ... elided
haskellPkgs = pkgs.haskell.packages.${compiler};
in haskellPkgs.developPackage rec {
root = ./.;
# ... elided
}If you look at the source it's because root eventually gets passed to src.
Likewise, this can be fixed with root = pkgs.nix-gitignore.gitignoreSource [] ./.;.
However, in that case the name has to be provided (e.g. name = builtins.baseNameOf ./.;) because the default implementation of name uses builtins.baseNameOf root which will result in a store path and give an error: "is not allowed to refer to a store path"
Here is the full (fixed) default.nix where I ran into this issue:
{ compiler ? "ghc865"
, doBenchmark ? false
, doProfiling ? false
, doStrict ? false
}:
let
sources = import ./nix/sources.nix;
pkgs = import sources.nixpkgs {
config.allowUnfree = true;
config.allowBroken = false;
overlays = [
(self: super: {
haskell = super.haskell // {
packages = super.haskell.packages // {
${compiler} = super.haskell.packages.${compiler} // rec {
ghc = super.haskell.packages.${compiler}.ghc // {
withPackages =
super.haskell.packages.${compiler}.ghc.withHoogle;
};
ghcWithPackages = ghc.withPackages;
};
};
};
})
];
};
haskellPkgs = pkgs.haskell.packages.${compiler};
in haskellPkgs.developPackage rec {
name = builtins.baseNameOf ./.;
# Must filter root otherwise `src = ./.` and lorri will get its known issue
# where it rebuilds on any file change
# (https://github.com/target/lorri/issues/6). When the root is filtered we
# must provide a name (above) because the default implementation uses:
# `builtins.baseNameOf root` which results in a name that refers to a store
# path and end up with the error: "is not allowed to refer to a store path".
root = pkgs.nix-gitignore.gitignoreSource [] ./.;
modifier = drv:
pkgs.haskell.lib.overrideCabal drv (attrs: {
buildTools = (attrs.buildTools or [ ]) ++ [ haskellPkgs.cabal-install haskellPkgs.ghc ];
enableLibraryProfiling = doProfiling;
enableExecutableProfiling = doProfiling;
testHaskellDepends = (attrs.testHaskellDepends or [ ])
++ [ haskellPkgs.criterion ];
inherit doBenchmark;
configureFlags =
pkgs.stdenv.lib.optional doStrict "--ghc-options=-Werror";
passthru = {
nixpkgs = pkgs;
inherit haskellPkgs;
};
});
returnShellEnv = false;
}
I have a little postgresql state dir inside my project's dot files, and when PG is running, it does changes some files regularly
It causes regular direnv reloads, like every 10 seconds and direnv debug log on each trivial command.
Is there way to exclude files from watching, probably even use
.gitignore?