Do not recommend experimental CLI in Nix Manual

kolloch commented 4 years ago

Describe the bug

The current "nix" CLI is still marked as experimental and this is not an accident: #3567

Therefore, it should not be needed or recommended for any non-experimental workflow but currently it is, e.g. in the Nix manual: https://nixos.org/nix/manual/#chap-distributed-builds

What I would like

The manual should explain how to achieve the goals without experimental CLIs.

OR

Parts of the nix CLI should be marked as stable.

matthewbauer commented 4 years ago

This is not only a problem in the Nix manual but also a problem with tons of other documentation that uses the new Nix command. I think we should make it opt-out instead of opt-in (https://github.com/NixOS/nix/commit/aabf5c86c9df1b4e1616a4cf06ee14a6edf2e5e1#commitcomment-35596337).

edolstra commented 4 years ago

What's the point of making it opt-in? The reason for putting it behind an experimental feature flag is that people don't use it accidentally.

LnL7 commented 4 years ago

I think it's a great idea to guard experimental features behind experimental flags rather than eg. keeping them on a long living branch. However at this point the nix commands have been released for long enough that this is going to cause a large amount if issues if released as is. At this point most people that run into this just add that setting, which only delays the problem it's trying to solve.

What's the exact reason this was added for the nix commands? If possible relaxing this a bit would really help to reduce the problems it will cause.

Only disallow specific commands that will most likely have behaviour changes, for example I don't think nix repl will really change. The commands that don't have an old counterpart are much more likely to cause problems.
Alternatively if this is eg. only for installables make only that part experimental so any other usages like -f or a drv are still allowed (assuming that will not change)

domenkozar commented 4 years ago

I share @LnL7 concern.

Another way to mitigate this annoyance is for message to explain how to quickly fix it.

zimbatm commented 4 years ago

The Nix 2.0 CLI was clearly marked as experimental. In my opinion, if people or part of the community started using it then they should be ready to pay the price when things start breaking.

kolloch commented 4 years ago

The Nix 2.0 CLI was clearly marked as experimental. In my opinion, if people or part of the community started using it then they should be ready to pay the price when things start breaking.

I disagree for those commands that have been advertised in the manual. It is easy to copy recommended commands from the Nix manual without realizing that these are experimental commands. You never see the experimental note.

The whole workflows should be marked clearly as experimental or exclusively non-experimental commands should be recommended. When you recommend commands in the manual without marking them as experimental at that place, you make an implicit commitment to the user that this will not set them up for failure. Even marking them as experimental and not providing stable alternatives would be rather unsatisfying.

If stability cannot be provided for those commands because it comes with a high long term cost, we should make the migration path as easy as possible. By e.g. detecting the use of old commands and printing the new command, providing clear migration instructions etc.

kolloch commented 4 years ago

I would even go further, for me personally, recommending some nix 2.0 commands in the manual, it made me think that they are ready to be used and the rest of the manual was simply not converted yet. That is obviously not true for everyone but I also cannot imagine that I am the only one.

edolstra commented 4 years ago

I agree they shouldn't have been used in the manual. That's why the experimental-feature option was added: it makes it harder to forget that a feature is experimental.

kolloch commented 4 years ago

Maybe, the experimental feature would have been great to manage expectations two years ago. Now, no perfect solution comes to mind. Maybe you need to introduce a new binary name like ʼn, ornix 3`.

Additionally or instead of this, bIt feels like you need to increase the major version number to signal that large of an incompatibility.

edolstra commented 4 years ago

Yes, bumping the major version number is a good idea, especially if flake support lands.

andir commented 4 years ago

Since the 2.3-branch is probably the most widely deployed Nix version(?) we should probably add a warning to all the users of the experimental command. E.g. by spitting out a (red/yellow/orange/…) warning whenever one of the experimental commands are being used? That should give those users at least a heads-up for when things will start to break.

LnL7 commented 4 years ago

The Nix 2.0 CLI was clearly marked as experimental.

That's true, but I think it's important to distinguish between intent and reality here. Most nixpkgs maintainers probably know it's not stable but even there usages sneaked in into various places, including nixos activation!

To reiterate I don't think keeping it completely stable is the way to go either, adding weird UX or a third version of commands to keep everything compatible doesn't seem worth it to me. However even just allowing a few stable commands commands already elevates a bunch of unnecessary pain for users without much downside.

deliciouslytyped commented 4 years ago

I think a different way to say this would be that the current nix interface has become de facto stable. I don't know enough history to think too hard about it, but it may be informative to consider why it happened.

stale[bot] commented 3 years ago

I marked this as stale due to inactivity. → More info

kolloch commented 9 months ago

This issue is three years old and with the 2.19 release apparently the nix flake lock --update-input command broke. Not a huge deal but an avoidable annoyance. Probably, there have been other papercuts like this before.

It is not even mentioned as breaking change in the release notes: https://nixos.org/manual/nix/unstable/release-notes/rl-2.19

Show love to your users and do not just remove a command from one release to another one.

Personal: Three years ago the reaction to this issue disappointed me so much that I withdrew from the nix community pretty much for a while. Now I am back and want to be back but I would like to see a more careful approach to changing the CLI interface.

What percentage of users have the experimental-features "nix-command" and "flakes" enabled? Do you really want then to experience breakages if avoidable? Most users cannot use nix in a sane way without "experimental-features".

The idea of relying on experimental-features to expose some new features for experimentation has failed in my opinion because the path to stabilization in nix is too long.

I propose:

All breaking changes, even in experimental interfaces, especially long available ones need a good reason.
The bar for the "good reason" for experimental features should still be much lower than for non-experimental ones.
But the breaking change and the reason should be elaborated on in the release notes.
The old behavior should be retained for a couple of releases if at all possible, printing a warning and a link to the reasoning.

I think that allows evolution without breaking user experience unnecessarily.

Note: I do not oppose cleaning up the CLI as the change I was referring to. But it should be performed with care for the users.

kolloch commented 9 months ago

A team member told me that the nix flake lock --update-input was a mistake and that there will be a backport of a PR that allows the old behavior with a warning. That is awesome :) Mistakes happen.

Ericson2314 commented 9 months ago

https://github.com/NixOS/nix/pull/9449

NixOS / nix

Do not recommend experimental CLI in Nix Manual #3569