Change argument parsing to a command/sub-command model

[jwiegley]

As Nix did with 2.0, rather than using lots of options and trying to emulate the UI of nix-instantiate, move to a command/sub-command parsing model, with each subcommand in its own file inside main.

[Anton-Latukha]

Moved from the :arrow_up: mentioned #737:

Essentially it is opinionated expansion of the above idea/mention into the pro argumentation for it:

---

HNix currently goes the way which Nix initially went - a set of --keys.

Which is nice and dandy, but people do not understand and do not know what options have what defaults and default behavior, and what can be combined, what needed to be combined, what should be combined and what happens when they combine:

# "How I enter the REPL?":
hnix --help
  --repl                   After performing any indicated actions, enter the
                           REPL
# What? what action? Can I just enter REPL?
hnix --repl
Welcome to hnix 0.10.1. For help type :help

hnix>
# Ok, seems I can, but that man message was puzzling, what they meant by that?

###

# Ok. Now next step. How I load an expression into a REPL?
hnix --repl '{}'
hnix: {}: openBinaryFile: does not exist (No such file or directory)

### That did not worked as expected.

echo '{}' | hnix --repl
Welcome to hnix 0.10.1. For help type :help

hnix> {}
hnix> Goodbye.

### Ok, that behaves differently, maybe it does what I want, but somehow exits the REPL.

### User goes RTFM:
hnix --help
  -E, --expr ARG            Expression to parse or evaluate

# Ok. Lets try --expr for expression.
# Lets try:
hnix --repl --expr '{}'

{}Welcome to hnix 0.10.1. For help type :help
hnix>
# Lets imagine that user knows that expression loads as `input`
hnix> input
While evaluating at <string>:1:1:
Undefined variable 'input'

# Ok, what's happening? I explicitly put to REPL that is an `--expr`.

# Probably even for devs I should remind here that `hnix --repl --expr '{}'` - pretty prints '{}' to output, and then loads clean REPL, but somehow we should explain user why --repl --expr ... does not evaluate expression into the REPL. "WTF" moments mean bad code/design.

# 
hnix --eval
Welcome to hnix 0.10.1. For help type :help

hnix> 

### Why Eval loads REPL? Is --eval equivalent to --repl? then why we need `hnix --eval --repl --expr '{}'` to even load any expression into REPL:
hnix --eval --repl --expr '{}'
{}
Welcome to hnix 0.10.1. For help type :help

hnix> input
{}
hnix>

# In other words, global defaults often do not fit all cases. Default modus operandi for REPL is to launch and load/evaluate/print stuff in it, not to print stuff before it launched.
# Eval is eval, if input is empty it is either Nothing in return, or error that there was no expression to eval, eval should not open REPL suddenly.
# For REPL or Eval the supplying of --expr is superfluous, what else REPL or Eval expect to get and load? Expression or file that contains expression, --expr is superfluous.
# ..etc

Nix use spawns a lot of --keys and explodes CLI and its complexity. If that is not handled - the result is a mash of keys, possibility of combination and behavior in a combination of which is unknown to the user. User can not learn this design of the CLI, since there is no design - it is essentially chaos of commutative --keys where it is not understandable what ordering of actions commutative keys result into, there seems to be a secret global ordering of those keys and actions, and secret internal rules on what set of keys result in what behavior. And the only thing for user is to be rigorous enough, is to hope to one day you sit and read all documentation at once and hope that all that documentation is so great that every cross-behaviour is got included, mentioned, described, explained and drawn attention to, and a hope that on reading all that you remember it all fully. Of course the puzzling model produces puzzlement, and people become shy to ask clarification, because some things are not put/understandable clearly in the documentation for them, and they fear to get rubbed directly into that or in some random documentation part or into "experience you should have" and "obviousness".

Nix has difficulty organizing all options to be easily followable, learnable, and usable: That is why there is:

nix
nix-build
nix-channel
nix-collect-garbage
nix-copy-closure
nix-daemon
nix-env
nixfmt
nix-hash
nix-info
nix-instantiate
nixops
nixos-build-vms
nixos-container
nixos-enter
nixos-generate-config
nixos-help
nixos-install
nixos-option
nixos-rebuild
nixos-version
nix-prefetch-git
nix-prefetch-url
nix-shell
nix-store

Because Nix used the current model the CLI HNix uses: "everything through --key". And then to simplify that design and use - lets split those into nix* executables.

The result is - executables have different CLI APIs, different output, and people forget Nix commands and get lost in them and never manage to understand them and so learn and know them all at the same time.

I do not think that HNix should follow the Nix CLI model, because Nix itself created that CLI mess in the first place, we must not copy it, but make a better design.

Basically that happened because from the start Nix avoided the model:

command action <args>

Nix currently tries to reverse-adopt this model:

nix --help
...

Available commands:
  add-to-store     add a path to the Nix store
  build            build a derivation or fetch a store path
  cat-nar          print the contents of a file inside a NAR file
  cat-store        print the contents of a store file on stdout
  copy             copy paths between Nix stores
  copy-sigs        copy path signatures from substituters (like binary caches)
  doctor           check your system for potential problems
  dump-path        dump a store path to stdout (in NAR format)
  edit             open the Nix expression of a Nix package in $EDITOR
  eval             evaluate a Nix expression
  hash-file        print cryptographic hash of a regular file
  hash-path        print cryptographic hash of the NAR serialisation of a path
  log              show the build log of the specified packages or paths, if available
  ls-nar           show information about the contents of a NAR file
  ls-store         show information about a store path
  optimise-store   replace identical files in the store by hard links
  path-info        query information about store paths
  ping-store       test whether a store can be opened
  repl             start an interactive environment for evaluating Nix expressions
  run              run a shell in which the specified packages are available
  search           query available packages
  show-config      show the Nix configuration
  show-derivation  show the contents of a store derivation
  sign-paths       sign the specified paths
  to-base16        convert a hash to base-16 representation
  to-base32        convert a hash to base-32 representation
  to-base64        convert a hash to base-64 representation
  to-sri           convert a hash to SRI representation
  upgrade-nix      upgrade Nix to the latest stable version
  verify           verify the integrity of store paths
  why-depends      show why a package has another package in its closure

Note: this program is EXPERIMENTAL and subject to change.

But it has difficulty to catch-up with all CLIs and their behavior and migrate tools back into the main organized CLI interface.

Adopting this CLI model early allows a much stronger organization of possibilities, allows to direct people with showing them only relevant available options with documentation of subcategory they chose, and also gives good completion help, overall that gives much better usability.

In optparse terms that submenus are subparsers, that is how commutative keys get organized for a particular purpose, and that purpose context allows to fit keys to expected behavior.

Great if HNix would be able to provide for HNix projects the plugin interface to add their subcommands into a hnix. So they can have their hnix-shell if they want, but also give hnix shell. That would integrate tooling and also "encourage" the tooling to be uniform between one another, main interface and practices get developed more, tools get easier to learn and use, they docs and maybe even complettions are in one menu, since they are uniform and so users have a much better user experience. And if they do not want to integrate into HNix CLI - so be it also.

[jwiegley]

I'm not entirely sure what the proposal here is: just to change hnix --repl to hnix repl?

[Anton-Latukha]

Pretty much.

I already pointed my thoughts above, answering the question - I would essentially repeat what already put.

An important point to see - is that people expect (for them the expected behaviours are obvious) particular way of operation for particular modus operandi. And that use of plain --options, creates a unstructured doc, "everything in one article", like Nix* manuals. This is a corner arguments that connetc/lead to all other, while comparing a single parser witth commutative set, and a submodes with subsets.

First (single set) - has cardinaly where it is impossible to cover all cases - and it would grow standard arg parsers significantly. If arg/behaviour has 1 default value/way - it often does not fit some modes of operation, and using different defaults when different set of args is supplied - is very obscure and troublesome behaviour. And creates a situation when the tool expects required superflous keys (hnix --eval --repl --expr '{}')...

Createing particular modus operandi, where subset is what allowed/sane for it with obvious keys simply not needed (like `expr), with unique keys and with operational defaults that are useful for particular mode of operation. Of course that would require the expantion of the subsets to what users require/what are the sane ways of the use modes, but parser reuse in Haskell is trivial.

This is no small task.

Would drag it for some arbitrary mileage, of course. It is a good entry point for learning whole project through code around it.

[Anton-Latukha]

Nix has a CLI guide now, moves to nix <subj> <verb> model, aka nix expr print.

I did some work for CLI in HNIx and understood that it is impossible to keep the two CLI systems at the same time, optparse-applicative design & Haskell record field are pretty coupled/limiting design and do not allow use abstractions to use in two CLI systems at the same time, through practice, I arrived at conclusions: Way 1. Introduce the a new CLI directly, breaking the old CLI into the new form. Way 2. Create a separate executable and develop new CLI there.

Currently I thought about the 2-d way. Lately, I am interested in ways that preserve the backward compatibility in Haskell, and because it is a path of the least resistance received from everybody.