search

c-blake / cligen

Nim library to infer/generate command-line-interfaces / option / argument parsing; Docs at
https://c-blake.github.io/cligen/
ISC License
501 stars 23 forks source link

Switching off option normalization #158

Closed sschwarzer closed 3 years ago

sschwarzer commented 3 years ago

I haven't been able to find out to how switch off the option normalization in dispatch*. I saw that I could use the parseopt3.getOpt iterators and just not call optionNormalize, but that's a much lower-level API compared to dispatch*.

If option normalization can't be switched off in dispatch* currently, could you please add this?

c-blake commented 3 years ago

No. There is no way to switch off normalization. It does not make a ton of sense for a CLI interface often used at an interactive prompt to be more strict than the prog.lang. Don't get me wrong..I also dislike Nim's general insensitivities..but the basic way the CL option keys are matched up with proc parameters and this feature raises follow-on questions and complexities that you may not have considered.

A short probably incomplete list is: Which would even be the "official speLLing"? The one in the proc parameter list or the one in the help key? I use the help key for the help messages. In this mode do we allow flexible matching between the proc and help and just restrict user input? Or are we strict everywhere? I.e., how much is your request about the CL user experience vs. the CL author development experience? When we do subcommands are we also strict about their names? And if so, what becomes of the mapping to APP_SUBCMD environment variables? Before I can say "yes, I'll add this", it requires a pretty thorough look at all the ways normalization is in use implicitly and explicitly.

It might help if you are more specific about what you think the above behaviors should all be or describing your situation in a bit more detail.

c-blake commented 3 years ago

I should perhaps have also said, if the broader agenda here is to force a less flexible CLuser experience, that the unique prefix matching of long option keys feature is more amenable to not always being on. Prefix matching is only about routing less specified user input to more specified program identifiers, a more sensible direction to deactivate than your desire here to route "more specified" user input to "less specified" program identifiers. The harder questions that arise really come from user input being more specified than what the compiler honors and the key being used in >1 place (input & output, at least).

Prefix matching is also available in Python, but evidently unpopular to some. Toggling it would fit well into the ability of clCfg / $HOME/.config/cligen to control (the way the [syntax] config file section can let end CLuser control reqSep and sepChars). So, I had already been thinking of adding a way to turn that off independently of this request.

It is true that the extra dash-collapsing optionNormalize makes CLuser input less specified. I.e., Nim idents can be disambiguated (with backquote stropping) by varying internal dashes. You were not specific enough about your situation/wants for me to guess if that is the issue. Case-underscore-past[0]-insensitivity are usually the more popular things to complain about and in the CLoption keys space, --kebab-case is I think the dominant preference.

Anyway, if we can make full sense of your request, it would also be a good candidate for that [syntax] section to be under end CLuser control (at least potentially). { And possibly for cligen to first look for ~/.config/cligen/progname/config before ~/.config/cligen/config so that per-progname overrides are also possible, but that may be nice anyway. }

sschwarzer commented 3 years ago

@c-blake , many, many thanks for your extensive answer! That's super-nice. :-)

Indeed I wasn't aware that the matter was so complicated. I'll try to answer some of your questions from my perspective, i. e. how I'd like to use the functionality:

Thanks for reading. I hope these explanations help a bit. :-)

c-blake commented 3 years ago

I'm not too eager to write extra documentation for explaning how it works. :-)

Every cligen command always has (no way to disable) --help-syntax (or --helps using normalization) which does explain the insensitivity as literally the 3rd bullet point. Can I ensure that a user who is surprised some feature works runs with --help/-h to see that --help-syntax is even a possibility? Nope. You can lead a horse to documentation, but you cannot make them read. ;-) But I can (and already do) save you that work. :)

Usually, the options for the user and in the help text should be all-lowercase

I was insufficiently clear. Currently, and for a long while, the CLauthor can decide how the option keys are rendered in the help text with the string used for the key in help. So, if you say dispatch(myProc, help={ "nAm-E": "usage message" }) then it'll use the ugly spelling. This is where the >1 use point causes complexity (for both me, the meta-CLauthor and CLauthors, but probably not CLusers). This feature is mentioned in 1st paragraph of Token matching in README.md, which I know is long (but still incomplete!).

didn't find information on the APP_SUBCMD

Well, it's technically kind of an opt-in feature that CLauthors can totally override, but it is mentioned at the bottom of the README.md in the final code snippet. It's just one more use case of the string which right now is "insensitive-ish" but would become more finicky in your requested mode. A subcommand name/proc could be camelCase. UPPER_CASE is typical convention for e.vars. But in a case-sensitive world, which would it be? Something for the CLauthor to scratch their head about and decide on, maybe, instead of just toUpperAscii.

may no longer be unambiguous if more options

Well, I do trap that as an error and the spell check/suggestion logic almost surely points out the right choices in such cases, but yeah..retraining fingers is unavoidable. :-) Anyway, as mentioned I am all-for abbreviation as a thing CLauthors can turn off and maybe be overridden by CLusers in a config file. { That could be another issue, but I won't forget and mentioned it myself. :) } CLauthors can technically override config parsing and block it 100%, but I doubt anyone wants to be that draconian (but they'll be able to be...).

To be honest, I don't understand this paragraph. Maybe it's too abstract for me.

It's mostly about keeping the mapping between Nim code and the CLI simple to describe to CLauthors (who are often their own primary CLusers), but it also has some "how hard is it to add to cligen" implications. Right now I can just say, "it's like Nim idents, but even more flexible with kebab-insensitivity" or the 3rd bullet in help-syntax. But if the CL is more restrictive than the code, well then I have to explain to CLauthors which is the official spelling - the code/API being wrapped or help, almost surely leaning toward the help since that is what is shown to users. There was recently even one very long-time user who didn't read/forgot about help entirely even for the per-parameter desc values even though it's the 2nd thing in READMEmd.

So, at a minimum, this becomes a complex thing to explain to other CLauthors to satisfy your evidently of-two-minds desire to disallow users casing/kebabing input flexibility (e.g. they may be required to say --timeout not --time-out) when you can already force all-lowercase-kebab output spelling in the usage message via help keys { which I would be $$ you didn't know about before this exchange and may be all you need to close this issue! :-) }.

switching to another library

Not to be rude, but making it easy to switch to other libs is kind of "out of scope" at least for me, but probably for anyone. That would be some mode or enough mode bit switches that can reach the lowest common denominator of every CL library out there which is hard-to-impossible to know, not very fun to find out, and probably syntactically impossible. Beyond that, shall we disable spell check lest users get used to the nice features and complain when you switch? I'm not a paid vendor (hah! I wish!), but anyone wanting to switch to some other for-free lib also has recourses to request whatever missing feature from there or fork and add themselves. I think cligen is great, but no CL library can be all things to all CLauthors along the full spectrum of all possible features. ;-) I realize you didn't say that, but it's logically very close to being directly implied by this portability idea. Anyway, that's just not a particularly persuasive argument.

c-blake commented 3 years ago

and not tell the user about the flexibility in writing the options. :-)

Oh yeah, this is also already possible, too. I think if you mkdir somewhere/cligen; cp cligen/syntaxHelp.nim somewhere/cligen and then ensure "somewhere" is ahead of wherever cligen lives (nimble or a git clone or wherever) in your nim module search --path that you can completely replace that message and delete that 3rd bullet point about insensitivity. I put it in its own file on purpose to allow that sort of Clauthor hacking, although I realize it's not the most CLauthor friendly "interface".

sschwarzer commented 3 years ago

You can lead a horse to documentation, but you cannot make them read. ;-) But I can (and already do) save you that work. :)

:-)

didn't find information on the APP_SUBCMD

Well, it's technically kind of an opt-in feature that CLauthors can totally override, but it is mentioned at the bottom of the README.md in the final code snippet.

Do you mean the snippet with the proc mergeParams? I don't understand how this interacts with subcommands. Anyway, I didn't find information on APP_SUBCMD because I searched in the page for APP_SUBCMD, then APP_ and didn't find anything. :-)

So, at a minimum, this becomes a complex thing to explain to other CLauthors to satisfy your evidently of-two-minds desire to disallow users casing/kebabing input flexibility (e.g. they may be required to say --timeout not --time-out) when you can already force all-lowercase-kebab output spelling in the usage message via help keys { which I would be $$ you didn't know about before this exchange and may be all you need to close this issue! :-) }.

You'd lose. ;-) I had read about help before but wondered if/how this could be enforced on the input side.

switching to another library

Not to be rude, but making it easy to switch to other libs is kind of "out of scope" at least for me, but probably for anyone. That would be some mode or enough mode bit switches that can reach the lowest common denominator of every CL library out there which is hard-to-impossible to know, not very fun to find out, and probably syntactically impossible.

I was only thinking about the "you can spell options however you like" feature, which is really special in cligen (as far as I know other CLI libraries). (Edit: I just had a look at the syntax help and saw that there are many more special features I didn't know about.) It would be nice feature if I could deactivate the feature ;-) , but I didn't mean I expected ("expect" in the sense "being disappointed about the 'lack' of the feature") the possibility to deactivate the flexible spelling. Of course it's your library, and you're free to implement or not implement what you like. :-) And cligen has so many features already that I totally understand that you don't want to add something that isn't trivial and not important to most users of the library (and probably not fun to implement).

Oh yeah, this is also already possible, too. I think if you mkdir somewhere/cligen; cp cligen/syntaxHelp.nim somewhere/cligen and then ensure "somewhere" is ahead of wherever cligen lives (nimble or a git clone or wherever) in your nim module search --path that you can completely replace that message and delete that 3rd bullet point about insensitivity. I put it in its own file on purpose to allow that sort of Clauthor hacking, although I realize it's not the most CLauthor friendly "interface".

Wow, I didn't think of that. :-D I probably won't go that far because it feels so hackish. :-) But nice to know it's possible. I wonder if it could be made easier to replace / change the syntax help, but I admit that's a very special "requirement".

As far as I'm concerned, we can close this ticket because my understanding is that the effort to implement "my" feature would be out of proportion with respect to its usefulness. Moreover, by modifying the help output I get most of what I'd like. Yes, the feature would be nice to have, but again, I totally understand the priorities. :-)

Thank you again for your replies and of course for cligen, as well as your other libraries and your answers in the Nim forum. I appreciate this! :-)

(By the way, I continued work on the "modify while iterating" tree iterator, but stopped the development for now because it became rather difficult and I have my doubts that the code could be turned into something generally useful. So I decided to spend the time on hopefully more useful Nim stuff. :-) )

c-blake commented 3 years ago

I don't understand how this interacts with subcommands

I guess that mergeParams example refers back to the proc bar(yippee:int ..) bit and presumes filename.nim=multi.nim or cmdName="multi" (not "app") and then has a regex $MULTI_(FOO|BAR) just in a side comment relating the proc names/subcommand names/e.vars. Yeah..Not so clear. Sorry. :)

really special in cligen[..]many more special features

I agree Nim's style insensitivity is very special. I also dislike it. Still, since the concept/sales pitch/high levelness of cligen is that "the host language is the CLI spec", it has more burden to play well with its host language conventions. And like you say, there's a ton of specialness. Most of it could be overridden/turned off, though. But not short option -f=val which someone over on a recent hackernews thread would complain about.

Anyway, there's a real "Do What I Mean; many ways to say stuff"/permissive vibe to the syntax. In my experience CLusers are far more likely to get frustrated when they try something and it doesn't work than to complain that the CLI is "not picky enough". You might be right that adding a more strict mode would be popular, but you're the first to ask. If >1 CLauthors ask I would probably give in or if you did a pull request, but I'm closing this for now.

easier to replace / change the syntax help

I agree it's hackish. I could add it as a parameter to clCfg which is by default initialized from that current file. It's one honking huge string parameter, but that would even let people, in principle, put it in their own ${CLIGEN:-$HOME/.config/cligen} to add private notes to. We could probably colorize it with render as well. Many don't read it, never mind want to change it, but that change is easier. That could be another issue along with abbreviation deactivation.

and your answers in the Nim forum

You're welcome. I usually think people just want me to shut up. I seem to have annoyed jibal yesterday who's got enough time/will to complain about some API, but not enough to discuss or even raise a github issue.

On that recursive iterator topic, I was recently trying something tricky (what FP people call a Z combinator). That's where you create recursion from non-recursion by applying a function to itself as a parameter. So, this almost works but Nim cannot handle the type (not the dynamics of the calling which I think would work fine and synthesize a recursive iterator):

type # illegal recursion in type
  RecItr = iterator(x: openArray[int], rest: RecItr): seq[int]

iterator rec(x: seq[int], rest: RecItr): seq[int] {.closure.} =
  if x.len > 0:
    let here: seq[int] = x[0..0]
    yield here
    let itr = rest(later, rest)
    for it in itr():
      yield it

It also probably would not solve your real problem. I think you would probably want an "edit cursor/Path"-like construct like the B-tree in adix for "general editing", but even then I'm not sure how easy it is to keep the cursor valid across various edits. I did it for B-trees, though. :)

I've been meaning to poke Araq at whether like "linked list" types that are self-referential in a type block we could allow this to then maybe silence the recursive iterator request crowd (maybe just me).

c-blake commented 3 years ago

I am cross-linking this to https://github.com/c-blake/cligen/issues/99 which (at least at the beginning) also discusses the problem before it devolves into Nim sensitivity in general and then jdupes vs. dups and dupes in general.

c-blake commented 3 years ago

Also, I have since re-learned that Python has done this long-option key abbrev forever which I think I once knew about but forgot. Oops. ;-)

sschwarzer commented 3 years ago

I agree Nim's style insensitivity is very special. I also dislike it.

I wouldn't say I dislike it, I'm more neutral. So far I haven't run into problems with it. I just use "camelCase" and "TypeCase" "like everyone else". ;-) On the other hand, I don't know how many libraries would actually be affected if the language feature didn't exist, so I would have to put up with mixed conventions across libraries.

But from my point of view the feature is even less useful when applied to command lines. ;-)

You might be right that adding a more strict mode would be popular, but you're the first to ask. If >1 CLauthors ask I would probably give in or if you did a pull request, but I'm closing this for now.

So I'll have to find more CLauthors to leave comments here. ;-) Regarding the pull request, that could be interesting, but probably not trivial and at the moment I have already too much stuff to work on. ;-/

[Syntax help]

I agree it's hackish. I could add it as a parameter to clCfg

That would be great! :-) As it happens I was checking the documentation and code more to see if I could override this somehow or leave the --syntax-help item out of the standard help. But I only found a way to disable the options help altogether, which isn't what I want. An auto-generated list for the options IMHO is one of the nicest features of a CLI library.

Regarding the other stuff which is no longer related to this ticket, I think we should communicate one-to-one on that, but unfortunately I have no contact data from you for that. As far as I can tell, you aren't on #nim in IRC, are you?

sschwarzer commented 3 years ago

Also, I have since re-learned that Python has done this long-option key abbrev forever which I think I once knew about but forgot. Oops. ;-)

The Python standard library has or has had three different libraries for command line parsing: getopt, optparse and now argparse. :-) Yes, I think argparse has always allowed abbreviations, at least by default. I don't remember the default for optparse. getopt used a lower-level API like parseopt in Nim.

sschwarzer commented 3 years ago

Repository owner deleted a comment from sschwarzer

That was quick. Thanks for your reply. :-)

c-blake commented 3 years ago

Yeah. Sometimes I'm super quick on the draw. ;) optparse also did abbrevs. Purportedly this feature was popular enough to do (people other than me think sloppiness is more motivated in command issuances) but unpopular enough to instigate whole new CLI libs. It's easy to disagree on either, both, but it's easier to turn off abbrevs without extra explaining. Maybe a simpler/less abstract way of explaining it is that the abbrev stuff is "only used in matching user input, never cross-referencing related entities". I added a couple items to TODO.md based on this thread. I should get to that stuff next week sometime. I'll tag you in the commit so you get notified.