Refactor the Resolver + CLI

[AntonC9018]

Don't merge this yet, I'm still refactoring the code.

Overview:

• orphanArgument token type, which means a positional argument after any named arguments.
• Tests for the binder module.
• Implemented groups for commands.
• Renamed and refactored some stuff.

More on the groups:

How it works is if there are any extra positional arguments, that argument will be treated as a nested command name. It will try to match any command that references the "command group" via a pointer field of that command context type. Before that nested command is invoked, the intermediate command is also executed.

The command tree or the command graph is created at compile time and is not available to traditional reflection at runtime. This can be added if needed tho. You'd just take all of the info on that graph (it's in an anonymous CTFE lambda) and just extract it in a template or something.

Currently, there is no way for the intermediate command to tell whether it's getting invoked as an intermediate command or as a terminal command, but I suppose we could just call a different method if it's not terminal.

Arguments are matched in this order:

• Until the minimum required number of positional arguments is reached, the arguments are fed into the command parser and are bound to the argument fields.
• Even if there still are positional optional arguments, we first try to match the next nested command from the list of the child commands of the current command type, then we try to bind the positional argument.
• If no positional arguments are possible, the arguments must be matched to a command, otherwise an error is produced.

Any of the help arguments -help --help -h --h /? /h /help are treated in a special way and currently result in a fix-me message being printed.

[AntonC9018]

@BradleyChatha can you disable the github actions so it doesn't work in vain?

[BradleyChatha]

Actions are disabled for now

[AntonC9018]

I think I'm done with this module for now. I've refactored it into a state machine. The code is very metaprogramming-heavy, though I'm sure it can be simplified.

The type graph thing is probably too compilcated, needlessly. Even though the sole purpose of mapping types to indices at compile time is so that the execution time of the algorithm is linear (it's quadratic if you'd just loop through the AliasSeq and find the type there), it can only matter if the number of types is significant. For a small number of types, a dumb version will be just as fast.

Another idea of how to map indices to types is via template resolutions, but this would rely on whether that is linear in the first place, which I don't think it is.

When immutable AA fields in templates become possible in D, that code can be simplified to just use those for the given mapping instead of restricting that info to CTFE or possibly converting it into a template name lookup hack via a mixin, if you happen to need those mappings in other places at compile time (commented-out code in groups.d, because we happen not to need it).

Things left to implement:

• Partial matches, pretty easy stuff. Just hook into the function that tries to match names (there is one for root names and another one for the subcommand names). Just modify that logic as you see fit. The only problematic bit is that you can't really uninline it, unless you do a mixin. There is a comment about this, but basically, you hit some pretty annoying frame issues.
• Command help, really easy, all of the info is there. You get the command type index and a notification that the help has been requested (see "special things"), in its simplest form, just print the help of that command once you're there.
• Removing false information from the readme, putting down the correct information, and also fixing the examples again.

I think it will be worth it to try and work on the API a little bit. In particular, using all the bits and pieces individually, starting from writing a small simple app and scaling up to an infrastructure. I think we should make sure the lib is easy to use at any scale.

[AntonC9018]

I have implemented a simple single-command API. See new_examples/most_simple.

[BradleyChatha]

Also I'll try to address any questions you've left in the code as well

[AntonC9018]

@BradleyChatha ping me here or on discord any time if you have any questions to me too

[BradleyChatha]

Will do :)

[AntonC9018]

I've cleaned up the code a little bit, and somewhat started on the type context idea. The internal hassle currently is that there are 4 different contexts to manage, which can get a bit confusing.

I've also just added an example where the commands are collected from the given module. The API level is at the easy to use and pretty complete level, even though internally, it is somewhat messy and still needs refactoring imo.

The help messages suck at the moment though, I hope you can get them sorted soon.

[BradleyChatha]

Just to be transparent, I'll probably take a look at the weekend now. I've had no energy to do anything this past week so I'm completely drained by the end of the workday T.T

Apologies, and I'll try to get this through as soon as I'm able to manage.

[BradleyChatha]

I think it will be worth it to try and work on the API a little bit. In particular, using all the bits and pieces individually, starting from writing a small simple app and scaling up to an infrastructure. I think we should make sure the lib is easy to use at any scale.

Need ideas to implement. Maybe something simple like reimplementing a lot of common command line applications, but in JCLI, to show it off a little.

One project I wanted to do with the original JCLI was to reimplement Dub's CLI, and see what pain points and blocked popped up, but never got around to it.

[AntonC9018]

@BradleyChatha I have done an example of simple usage and an example of module-based usage, since I left that comment. It's just that I think that if we claim our thing is modular and easy to use, we should prove it really is by trying to use it as such, which way we'll discover flaws and be able to adjust or refactor it accordingly.

Not necessarily big projects, demos like in your examples work, it's just that this should be done for each module that you think can be used stand-alone. Just do such demos and you'll discover things that feel out of place, something or extra fluff that the user ideally does not need to worry about.

Example: the old command parser (I think) had the extra requirement of a command UDA being there on the command to be able to use it. I think this is what's called a quirk because the user does not care about any of the info that may be provided via that attribute to the application, but it still needed to be present, which was annoying.

Another example: the old CLI only worked with modules, taking on command discovery, which feels like a burden when all you want is to execute a single command.

[AntonC9018]

Why do you need the test runner for those examples btw? Integration tests? Or just making sure it compiles? Because tests from the source code are plenty. I think demos are fine to stay as demos.

[AntonC9018]

If you think you could break something with code changes, then just add normal unit tests, I think that would be more beneficial

[AntonC9018]

We might still need integration tests for the module features, just because it's hard to fake it with unit tests, but you don't have to test all examples. For example, there is no benefit to testing the tokenizer like that, because it can be easily unit tested in isolation.

[AntonC9018]

Even then, I think handcrafted demos should be separate from actual rigorous tests.

[BradleyChatha]

@BradleyChatha I have done an example of simple usage and an example of module-based usage, since I left that comment. It's just that I think that if we claim our thing is modular and easy to use, we should prove it really is by trying to use it as such, which way we'll discover flaws and be able to adjust or refactor it accordingly.

Not necessarily big projects, demos like in your examples work, it's just that this should be done for each module that you think can be used stand-alone. Just do such demos and you'll discover things that feel out of place, something or extra fluff that the user ideally does not need to worry about.

Example: the old command parser (I think) had the extra requirement of a command UDA being there on the command to be able to use it. I think this is what's called a quirk because the user does not care about any of the info that may be provided via that attribute to the application, but it still needed to be present, which was annoying.

Another example: the old CLI only worked with modules, taking on command discovery, which feels like a burden when all you want is to execute a single command.

Yeah, it's just a good idea in general to 'practice what we preach'. The more examples we can come up with the better.

Why do you need the test runner for those examples btw? Integration tests? Or just making sure it compiles? Because tests from the source code are plenty. I think demos are fine to stay as demos.

It was mostly just to make sure that they actually compile. Too many times have I seen projects with broken examples. I didn't want to end up as one of them :D

Even then, I think handcrafted demos should be separate from actual rigorous tests.

Perhaps we can make the distinction between examples of module usage, and demos of a fully-fledged program. The former can be handled with unittests, the latter with a test runner.