Is there a way to use cligen in rpc style?

[geohuz]

Suppose I have a rpc server wraps several function calls inside it, and I want to use cligen to parse the command args submitted from clients and dispatch to corresponding procedures. What is the skeleton supposed to be?

[c-blake]

While I have never used cligen this way or heard of it being done, there are several existing mechanisms that might help, depending upon why you want this. Any would first need something to parse a network message string at a master RPC entry point into a seq[string]. The stock mergeParams() uses cfUt.envToCL() which uses the stdlib cmdline.parseCmdLine() which has kinda janky quoting rules.

Before detailing those ideas, I would note that most of the value (in subjective work-saved-units) of the basic cligen mechanism is "human-oriented" (either making options/args easier to keystroke in or getting nice, autogenerated help screens). Meanwhile, RPCs are typically "machine-generated" from "something else". So neither input CL-parse side nor output help side may profit much in an RPC context. The default control flow organization is also a bit.. misaligned with commands to an RPC server since things tend to quit the program on errors.

That said, it is true that cligen is mostly like an FFI to translate between seq[string] and native proc calling conventions. The most direct idea might be to use the mergeParams() hook to inject command args (options|parameters) from config files or environment variables or environment variables pointing to config files. Since you say you have "several" functions to wrap, the most salient example might be cligen/mergeCfgEnvMulti.nim.

Another idea is that since generated dispatchers just accept any old seq[string] of which a command-line is just one example (see https://github.com/c-blake/cligen/blob/4193f802796f15559c81c6dd56724d6f20345917/cligen.nim#L810-L812), you could just call them directly. You should look at how cligenQuit calls them, though (really from that part of the code all the way up to dispatch or possibly dispatchMulti which is far more complex). If you don't call it right, it might accidentally "exit your RPC server" with some error code on a malformed message which is, presumably, undesirable.

A third idea, closely tied the second, is to use the parseOnly mode of generated dispatchers. test/ParseOnly.nim has a kind of partial example/test, but you are in luck - bu/dirq.nim was recently changed to have a more complete example here.

A fourth idea is that maybe you only like the more "Unix like" option-parsing cligen provides. Then you might be able to use the cligen/parseopt3 mechanism directly after the "create a seq[string] phase".

If you try one of these and run into trouble, feel free to comment here to ask questions, but I am closing as this is far outside the generally intended application of the package. Honestly saying nimble search rpc turns up like 12 other packages that seem very likely to be more appropriate for your larger use case. To the extent most of the work is serializing/deserializing std/marshal and about 10 other JSON packages can probably help there. In a CLI setting, one is "stuck with text/string inefficiency" because that's what users keystroke in or the "help" they can understand, but for truly maximum efficiency in program-to-program communication 5 other "non-json but still cross-platform marshaling packages" (like "flatty") might be more appropriate. It probably depends upon problem constraints (e.g. must interact with dumb web browser JSON/etc.). IF you know anything about the pair of CPUs in question then you can fix on a "platform" (e.g. x86|ARM) and run off of raw CPU friendly memory buffers without even byte-swapping saving an entire "pass" over the data and working at raw memory-copy bandwidth which is usually 10+X faster. (You could have a little magic 2 byte number at the start of each client connection to detect incompatibility rather than crash and, hey, maybe someone someday will connect with some 2007 PowerPC OSX Mac someday - or maybe not.) BTW, all this above is just me trying to help as much as possible not "judge", but you didn't say a whole lot about your problem context and so as a result there is a very wide spectrum of possibility.

[geohuz]

@c-blake thanks for the detailed information! I'm trying to design a trading system with UI that users can submit order and commands like so:

myApiKeys(XBTUSD) {
    # Buy 100 contracts NOW
    market(side=buy, amount=100);

    # Add a stop loss
    stopMarket(side=sell, amount=100, offset=e1%, trigger=last);

    # We can just use a normal limit order as our take profit...
    limit(side=sell, amount=100, offset=e3%, reduceOnly=true);
}

I'm seeing the above commands a list which map to remote procedure calls, so with using cligen I can easily accomplish this by using cligen with benefits of args validation, and help text response utilities.

[c-blake]

If traders are submitting orders by keystroking commands in a terminal window then I think a dispatchMulti where the impls fire off the API calls could make sense.

On the other hand, if traders are filling out a form on a web browser, it sounds to me like what you might want is a "SPA gen" (single-page application) not a "CLI gen". I discussed this briefly in this comment here 6 years ago.

It sounds to me like the second interpretation is most likely. So, maybe all you want to do is leverage cligen/argcvt which does the argParse-argHelp stuff? Learning how ArgcvtParams works (that dirq.nim code I pointed to already still stands as an example of that). Some hypothetical spagen could perhaps leverage cligen/argcvt, but the requirements may be just too different. For example, argcvt.argParse[T](dst: var seq[T], ..) concerns itself with "-=" modifiers based on an idea users may have put that in their command-line options, but that may make no sense at all for a web API call with a bunch of web form ...?a=b&c=d&.. parameters. cligen/strUt.tmplParse* may help pre-parse an HTML template for the page and along with std/cgi could perhaps create a UI layer.

Anyway, the short of it all is that while much in cligen kind of "sits at the edge" of needed functionality for what I think you are describing, nothing is really designed to make applying it easy in your context.

I think there are many more appropriate webUI helper Nimble packages that do try to make it easy. E.g., I believe there is a package by the Nim lead Araq called "Karax" used for the Nim Forum, but I have never tried to write a program with it. nimble search frontend turns up a lot, although you might find searching nimble packages.json easier with a smarter search tool.

[geohuz]

@c-blake thanks for the mindful and extensive discussion about this topic, now I need to rethink my idea.