Open systemofapwne opened 2 years ago
This is the expected behavior AFAIK
Summary: I'm not aware of a simple and easy solution to this UX annoyance that does not lead to more UX problems elsewhere.
This is intended. While making this new behavior default would simplify a few situations, it will also make others more complicated. As an analogy to another technology that purports to provide reproducible source code trees (git) that does not provide a similar default behavior:
1) git init test && cd test
2) copy any file in there
3) run git show
See error
Perhaps the error message should be better, or there should be an easier way than git --intent-to-add
to do what you'd like. Perhaps it should automatically include it, but with a warning that clarifies what is happening, but then we must be careful to avoid this becoming depended on and then authors accidentally leave files out of repos that are needed for evaluation to occur. Then all this behavior needs to be explained. For now, the rule is simple: "if you are in a git repo, flake evaluation only utilizes files that git is tracking."
Honestly the fact that flakes don't automatically add files is a feature, and for me it's one of the biggest draws because of its interaction with the eval cache.
Sure sometimes I forget to add a file, but I almost always forget to gitignore
things like result
in a fresh repo. If files were auto added without a flag or option, the eval cache would trigger rebuilds as often as nix-build
used to.
Summary: I'm not aware of a simple and easy solution to this UX annoyance that does not lead to more UX problems elsewhere.
This is intended. While making this new behavior default would simplify a few situations, it will also make others more complicated. As an analogy to another technology that purports to provide reproducible source code trees (git) that does not provide a similar default behavior:
- git init test && cd test
- copy any file in there
- run
git show
See error
I can understand, that this is by design and intended. I can even live with that decision. Regarding your next statement...
Perhaps the error message should be better, or there should be an easier way than
git --intent-to-add
to do what you'd like.
I am absolutely for that. I spent about two hours debugging this and I am and will not be the last one falling for that. The error message should be way more clearer for a better UX. Like
error: getting status of '/nix/store/0ccnxa25whszw7mgbgyzdm4nqc0zwnm8-source/flake.nix': No such file or directory
Local flake.nix found, which is not member of the repository. Consider adding it first.
or something similar.
Honestly the fact that flakes don't automatically add files is a feature, and for me it's one of the biggest draws because of its interaction with the eval cache.
Sure sometimes I forget to add a file, but I almost always forget to
gitignore
things likeresult
in a fresh repo. If files were auto added without a flag or option, the eval cache would trigger rebuilds as often asnix-build
used to.
Maybe (or certainly) I do not understand the full nix architecture in detail yet.
But I was rather referring to making flakes not fail when a flakes.nix file certainly is there (but not a member of a repo yet). Not automatically adding it to the repo.
Yet, as stated one post earlier, I can live with that behavior/design choice. The error message should just be more clearer, so that someone without a deep understanding of nix, flakes etc can troubleshoot the situation with a simple git add flakes.nix
.
I have the very same issue. My use case is this https://nixos.wiki/wiki/Flakes#Super_fast_nix-shell
I am using this for repos that might also be used by non-nix users that would get annoyed by adding shell.nix
and flake.*
files to the repos.
In the old world, I had my shell.nix
in .gitignore
(no flake.nix
) and everything worked fine.
Also adding the flake.nix
file to .gitignore
will not be enough to get rid of "flake.nix': No such file or directory".
I have the very same issue. My use case is this https://nixos.wiki/wiki/Flakes#Super_fast_nix-shell I am using this for repos that might also be used by non-nix users that would get annoyed by adding
shell.nix
andflake.*
files to the repos. In the old world, I had myshell.nix
in.gitignore
(noflake.nix
) and everything worked fine. Also adding theflake.nix
file to.gitignore
will not be enough to get rid of "flake.nix': No such file or directory".
Ignoring flake.nix
would prevent your flake from ever being available. Definitely don't add that to your ignores.
I think you misunderstood my comment about how legacy UI commands would cause spurious rebuilds because unimportant files were added to the working dir. The common workaround was various source filters, notable nix-gitignore
. The old workflow was basically "clone a repo, extend .gitignore
( to the point that it's almost identical to git clean -xfd
), create *.nix
using nix-gitignore
filters, and THEN run nix-*
.
Skipping these steps would cause Nix to rebuild projects needlessly, often to such a degree that it was easier for Nix files to be maintained externally. The new UI'S git
integration resolves these issues. The tradeoff is "I don't have to do all that filtering, I just have to git add flake.{nix,lock}
once and forget about it". 😉
I am absolutely for that. I spent about two hours debugging this and I am and will not be the last one falling for that. The error message should be way more clearer for a better UX.
There is a pending PR which has a better message. What do you think about this? https://github.com/NixOS/nix/blob/5f1340219b83f15a4354aad94467ae642a1196ed/src/libfetchers/git.cc#L756-L759
I am absolutely for that. I spent about two hours debugging this and I am and will not be the last one falling for that. The error message should be way more clearer for a better UX.
There is a pending PR which has a better message. What do you think about this?
This is a welcome solution for me!
I've also lost several hours to this issue. Since it wasn't completely non-trivial to find, the PR that contains this change is https://github.com/NixOS/nix/pull/6530. That PR is also still a draft, so I'd prefer this to stay open until it's merged.
@tomberek / @edolstra could one of you reopen this issue and put a "Closes #6642" in the PR description for #6530?
Like @WhatisRT, I lost hours to this issue. @tomberek Please follow @WhatisRT's suggestion and reopen the issue. Thank you.
This situation might be improved by making the nix CLI default to path:.
instead of git+file:.
when flake.nix
is not added to git and/or is gitignored.
The usecase of adding a local flake.nix
to a repo which would not accept it being committed upstream would be streamlined this way, though it wouldn't stamp out the unintuitive issues related to files not existing due to being untracked or ignored by git. It might even add to them, since adding flake.nix
would change the behavior of other files in this regard.
That error message is indeed terrible UX, it provides no clue as to what's going on.
This situation might be improved by making the nix CLI default to
path:.
instead ofgit+file:.
whenflake.nix
is not added to git and/or is gitignored.The usecase of adding a local
flake.nix
to a repo which would not accept it being committed upstream would be streamlined this way, though it wouldn't stamp out the unintuitive issues related to files not existing due to being untracked or ignored by git. It might even add to them, since addingflake.nix
would change the behavior of other files in this regard.
Now this I actually agree with.
For clarity though you should add flake{nix, lock}
to your repo. Yes the error message could be better ( PR it ).
This situation might be improved by making the nix CLI default to
path:.
instead ofgit+file:.
whenflake.nix
is not added to git and/or is gitignored.
in many situations that would also lead to copying lots of data into the nix store, which i would find to be a regression. people would have to readjust their habits to explicitly state git+file.
which i certainly feel resistance towards (i.e. laziness :laughing:).
since in the case of a dirty git tree users already get a warning, i suggest to also print a warning if flake.{nix,lock}
exist but are not racked in git.
in many situations that would also lead to copying lots of data into the nix store, which i would find to be a regression. people would have to readjust their habits to explicitly state git+file. which i certainly feel resistance towards (i.e. laziness ).
It would only cause that in situations where currently it fails completely. When flake.nix
is known to git, it would still default to git+file:.
, just as it does now.
Moving from non-functionality to poor performance doesn't seem like a regression to me.
It would only cause that in situations where currently it fails completely. When
flake.nix
is known to git, it would still default togit+file:.
, just as it does now.
ah, i interpreted your suggestion as changing the default unconditionally. but you meant it to be dependent on the flake.{nix,lock}
tracking status, did i get it right now? if so it'd deem that acceptable, however still somewhat of a foot-gun. how about in that case nix just acts as if flake.{nix,lock}
where tracked?
Yes, that's what I meant.
Indeed, it might be better to just pretend flake.{nix,lock}
are tracked even when they aren't. I haven't thought it all through, but it does seem like that has less potential fallout, and avoids copying .git
to the store all the time, too.
This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/doing-the-zero-to-nix-and-get-a-random-error/27692/8
2 years since the issue creation and we still cannot run nix develop when the flake.* files are in gitignore? This is my company's app and do not want to commit these flake files which are not related to the project and which are only used by me. Just let me do what I want instead of imposing me this decision...
@jerlam06 have you tried nix develop path:.
?
The problem is that there isn't really a good solution here.
If nix copies non-git files generally, then you end up with all kinds of detritus and intermediate files copied into the nix store, and then those intermediate files often mess up build processes because the timestamp data is gone.
If you build only from the latest actual commit then it's too annoying to quickly test out a change to a flake.
The compromise we've landed on is to use the dirty copies of any files tracked by git, but ignore other files. It's not perfect, but it's quite hard to come up with a change to it that wouldn't cost too much in terms of other unintuitive results. I'd love to see a nice solution to the usecase of adding a local flake.nix to a repo that doesn't have one upstream, but any such solution needs be considered in terms of all the effects it would have, not just that it solves that one issue.
I don't think we should be forced to overload .gitignore
to serve this separate purpose of defining what goes into the nix store. Maybe have that as a default, but it needs to be overridable. I just tried using nix flakes for a new project for the first time and this is the very first use case I wanted. I think using flake.nix to nixify a project without having to check the flake.nix file into the repo is a very important use case.
If a flakes.nix file is copied to a repository (empty or with content), then without adding it yet to the reposiroty, envoking
nix shell
and other flake related commands will fail with an error likerather than falling back to just use the locally available flake.nix (should be the default IMH).
Steps To Reproduce
git init test && cd test
nix shell
Expected behavior
A nix shell should be opened using the locally present but not yet added or commited
flakes.nix
.nix-env --version
output nix-env (Nix) 2.8.1Additional context
The flake.nix file, that I used