Open jjr840430 opened 6 years ago
Thanks for this, @jjr840430
NOS syntax is still up for discussion, please make comments/suggestions. As an initial translation, something like:
option config {
short f
long "config-file"
usage "Specify configuration file"
hint FILE
env RVD_CONFIG_FILE
default "/opt/rvc/etc/rvd.conf"
}
option help {
short h
long help
usage "Print this help message"
default_arg true
default false
}
option version {
short v
long version
usage "Print version information"
default_arg true
default false
}
option go_daemon {
short D
usage "Run in daemon mode"
default_arg true
default false
}
option check_config {
short c
usage "Check configuration and exit"
default_arg true
default false
}
option openvpn_bin {
short o
usage "Specify OpenVPN binary path"
hint PATH
config "openvpn.path"
env OPENVPN_BIN_PATH
default "/opt/openvpn/sbin/openvpn"
}
option openvpn_root_check {
short r
usage "Enable root check for OpenVPN binary"
hint BOOL
config "openvpn.enable_root_check"
default_arg true
default true
}
option openvpn_updown_scripts {
short s
usage "Enable up/down scripts for OpenVPN"
hint BOOL
config "openvpn.enable_updown_scripts"
default_arg true
default false
}
option user_id {
short u
usage "Specify User ID for RVC process"
hint UID
env RVC_USER_ID
config "global.user_id"
default "-1"
}
option restrict_socket {
short r
usages "Restrict access to communication socket"
config "global.restrict_socket"
hint BOOL
default_arg true
default true
}
option log_directory {
short l
usage "Specify the log directory of RVD process"
hint DIR
env RVD_LOG_DIR
config "global.log_directory"
default "/var/log/rvd"
}
option vpn_config_paths {
short p
usage "specify the directory path of VPN configurations"
hint DIR
config "global.vpn_config_paths"
default "/opt/rvc/etc/vpn.d"
}
After processing each option generates exactly one value which is inserted into the configuration tree.
hint
is used for a concise usage message.usage
is for longer help information.default_arg
is a switch that may be used without an arg.config
field specifies the configuration node. If omitted the option's key is used instead.config
, help
and version
option config
specifies the configuration file to use.option help
and option version
don't generate a value; they display help and version information respectively-c
--config-file
, -h
--help
and -V
--version
. Adding a config
field to any of these special options will revert them to normal processing so the default behaviour can be overridden / suppressed.env
field)default field
)@drystone Thanks for your sharing your structure.
It seems your structure is better than one for libnereon
.
The remaining is how to define sub-options
.
I think we may take some example models for sub-options. I will post my thoughts shortly.
For example, rvc
has the following sub-options:
rvc edit <connection name> <auto-connect|pre-exec-cmd|profile> <value>
edit
is main option, and auto-connect
,pre-exec-cmd
, profile
is sub option for edit
.
Also edit
should have at least one suboption of them.
Then we may think the following NOS syntax for edit
option.
option edit {
short e
long edit
usage "Edit RVC VPN connection"
sub-option-required true
sub-option auto-connect {
long "auto-connect"
}
sub-option pre-exec-cmd {
long "pre-exec-cmd"
}
sub-option profile {
long "profile"
}
}
Perhaps we could call edit
a sub-command?
I think NOS could handle sub-commands but options without names may need to be handled outside of NOS. Or would this be acceptable?
rvc edit <--connection=NAME> <--auto-connect=VALUE|--pre-exec-cmd=VALUE|--profile=VALUE>
or
rvc edit <--connection=NAME> <--option=OPTION --value=VALUE>
The command line format for rvc
was suggested by @ronaldtse.
I'm not sure whether all command line formats should be handled by NOS configuration.
I think we need to define command line syntax too which is supported by nereon
.
@ronaldtse What do you think about this?
I think it's not possible to support all command line formats by nereon
.
So we need to define command line formats too with NOS configuration syntax.
I agree we should support subcommands. Let's first distinguish a "command" from an "option".
--{long-option}
or -{short-option}
For example:
git checkout -b <branch>
aws s3 ls <bucket-name> [options]
In the case of aws s3 ls <bucket-name> [options]
ls
is clearly a commandaws
, s3
, they maybe command-namespaces, or commands themselves? I'm not sure.<bucket-name>
is clearly an argument for the ls
command[options]
are clearly options that should be prefixed with --
.Commands are permitted more than one non-option argument:
ls dir1 dir2
cp file 1 file2 dir
These should be described in NOS so they can be inserted into the resulting NOC. Something like
args {
config "args"
}
Subcommands could be described as:
# common options for <progname>
...
subcommand one {
# common options for <progname> one
....
subcommand a {
# specific options for <progname> one a
....
}
subcommand b {
...
}
}
subcommand two {
...
}
subcommand three {
...
subcommand c {
....
}
subcommand d {
....
}
}
@drystone we don't need to separate command line option to command
and option
.
it may be compromised when implementing several types of the command line by NOS.
we may think command line
as the following syntax:
prog_name <option1> [args1] <suboption1> <> <options2> [args2] <suboption2> <>...
option1
and option2
is the item with statically defined names by the program.
it has the following requirements.
suboptions
suboptions
is the item with statically defined names by the program, but it can't be used without option
or other suboption
.
suboption
has the same requirements as option
.
args
is the item which is defined with arbitrary data so it's not defined in NOS configuration.
From my thoughts, let's take some examples:
rvc edit vpn1 --auto-connect enable
edit
is option
,
--auto-connect
is suboption
which may be used by only edit
option,
vpn1
and enable
is args
for edit
and --auto-connect
cp /tmp/1 /tmp2
this example doesn't have any option
and suboption
.
it has only args
, /tmp/1
and /tmp/2
git checkout -b new_branch
checkout
is option
-b
is suboption
new_branch
is args
for -b
aws s3 ls <bucket-name> [options]
s3
is option
ls
is suboption
of s3
bucket-name
is args
for ls
.
@ronaldtse @drystone
Here is NOS configuration example for rvc
.
Since rvc
doesn't have configuration file so I will post another example related to NOC configuration in the next comment.
suboption json_flag {
switch "--json"
type "boolean"
}
option connect {
type "string"
switch "connect"
desc "connect to a VPN with given name (default:all)"
hint "[all|connection name] [--json]"
default "all"
suboptions {
option json_flag
}
}
option disconnect {
type "string"
switch "disconnect"
desc "disconnect VPN with given name (default:all)"
hint "[all|connection name] [--json]"
default "all"
suboptions {
option json_flag
}
}
option reconnect {
type "string"
switch "reconnect"
desc "reconnect VPN with given name (default:all)"
hint "[all|connection name] [--json]"
default "all"
suboptions {
option json_flag
}
}
option status {
type "string"
switch "status"
desc "get status of VPN connection with given name (default:all)"
hint "[all|connection name] [--json]"
default "all"
suboptions {
option json_flag
}
}
option edit {
type "string"
switch "edit"
desc "edit VPN connection with given name"
hint "<connection name> <auto-connect|pre-exec-cmd|profile> <value>"
suboptions {
requires true
type "mixed"
option auto-connect {
type "string"
switch "auto-connect"
}
option pre-exec-cmd {
type "string"
switch "pre-exec-cmd"
}
option profile {
type "string"
switch "profile"
}
}
}
option remove {
type "string"
switch "remove"
desc "remove VPN connection (sudo required)"
hint "<connection name> [--force]"
suboptions {
option force_flag {
type "boolean"
switch "--force"
}
}
}
option import {
type "boolean"
switch "import"
desc "import VPN connection (sudo required)"
hint "<new-from-tblk|new-from-ovpn> <path>"
suboptions {
requires true
type "single"
option new_from_tblk {
type "string"
switch "--new-from-tblk"
}
option new_from_ovpn {
type "string"
switch "--new-from-ovpn"
}
}
}
option reload {
type "boolean"
switch "reload"
desc "reload configuration (sudo required)"
}
option dns-override {
type "boolean"
switch "dns-override"
desc "override DNS settings (sudo required)"
hint "<enable <DNS serve IP list>|disable|status>"
suboptions {
requires true
type "single"
option enable {
type "string"
switch "--enable"
}
option disable {
type "boolean"
switch "--disable"
}
option status {
type "boolean"
switch "--status"
}
}
}
option script-security {
type "boolean"
switch "script-security"
desc "enable/disable script security"
hint "<enable|disable>"
suboptions {
requires true
type "single"
option enable {
type "boolean"
switch "--enable"
}
option disable {
type "boolean"
switch "--disable"
}
}
}
option version {
type "boolean"
switch "version"
desc "print version"
}
option help {
type "helper"
switch "help"
desc "print help message"
}
Here is the description for NOS configuration of rvc
.
NOS configuration may have the following sections, suboption
and option
.
suboption
: define commonly used suboptions in option
. In the above example, json
suboption is used in connect
, disconnect
, reconnect
, status
options.
option
: define standalone options
Each field in option
or suboption
section has the following purpose:
type
: it has the built-in types for NOS configuration. it may have the following types:
boolean
: it doesn't require any argument
string
: the argument is a string
int
: the argument is an integer
array
: the argument is an array
float
: the argument is a float
config
: the argument is used to override configuration file
help
: the option is used to show help message
switch
: it specifies switch
how to specify that option. if the option is specified by two switch
options, then it may take the following format
switch "<switch1>|<switch2>"
for example: `<-l|--verbose>`, `<config|-config>`
desc
: it specifies the description of that option in help
messagehint
: it specifies the description how suboption or arguments should be given for that option in
help
message.default
: it specifies the default value of argumentsuboptions
: it specifies suboptions used by that option
this section has the following fields:
requires
: it specifies whether suboptions are mandatory for that option
type
: it specifies how suboptions are used for that option. it has signle
and mixed
.
single
: only one option from suboptions must be used for that option
mixed
: several options may be used at same time for that optionoption
: it specifies each suboption.Since NOS configuration is compiled with the main program, I suggest the following another section:
program {
name "<program name>"
version "<program version>"
copyright "<copyright message>"
homepage "<homepage URL>"
}
version
, copyright
and homepage
may be used for command line to show version info.
To integrate with NOC configuration, I suggest config
section in NOS configuration:
config <config name> {
type <configuration type>
default <default value>
env <environment variable>
requires <true or false>
subconfigs {
config <subconfig name> {
type <type of subconfig>
default <default value of subconfig>
env <environment variable>
requires <true or false>
}
}
}
config name
: it specifies the configuration name in NOS. it should be composed by only alphabetical characters, -
and _
.
type
: the type of configuration. it may have the following types:
object
: the configuration item has sub configurations
string
: the configuration has string type
integer
: the configuration has integer type
array
: the configuration has array type
boolean
: the configuration has boolean type
float
: the configuration has float type
default
: default value of the configuration
requires
: specify whether that configuration should be given in NOC configuration
subconfigs
: defines sub configurations belong into that configuration
If NOS has command line option to override the NOC configuration item, then it may be specified in option
as the following:
option <option name> {
...
config "<config path>"
...
}
config path
means the path of configuration item.
If configuration is subconfig
, then config path
should be specified by concating config section names
using .
.
Here is the example:
config global {
...
subconfigs {
config log_directory {
...
}
}
...
}
option log_dir {
...
config "global.log_directory"
...
}
I tend to agree with @ronaldtse that there is a semantic difference between options, subcommands and arguments. Options traditionally have the form -o
or --option
. Subcommands, if any will be the first n
non-option arguments and the remaining non-option arguments are simply arguments for the command.
These could all be treated the same but I think the NOS language would be more intuitive if it can describe/model these concepts more naturally.
As I said earlier, there is no concept of type in NOC syntax (nor in command line arguments). All leaf values are strings and it is the responsibility of the program to convert these into meaningful values and to error out if the configuration values can't be parsed. Therefore I don't think the type
fields are necessary other than to describe whether an option is optional an/or whether it requires an argument - and I think we can find a better language for this.
I'm off now but look forward to further discussion.
@drystone Thanks for your comment.
I also think that @ronaldtse 's opinion is meaningful.
but sometimes it may be confused when determine which command line argument is specified as subcommand
or option
.
if we may define command line arguments with static names as option
or suboption
, then it may reduce that confusing.
Also type
in NOS and NOC configuration is for checking whether configuration item has the correct format for built-in
types such as integer, string, boolean etc.
It's possible to check them on the main program side but I think it may reduce the coding lines if nereon
library checks the type of each arguments while initializing.
We may override type-checking function in the main program side for specific data types such as date, IP address or other formats.
@ronaldtse do you have any suggestions?
Since NOS configuration is compiled with the main program, I suggest the following another section:
program { name "<program name>" version "<program version>" copyright "<copyright message>" homepage "<homepage URL>" }
version, copyright and homepage may be used for command line to show version info.
This is a good idea @jjr840430. Maybe not name
as it's generally argv[0]
. Unless there is any occasion when you'd want to report any name other than the name used to invoke the program itself?
I think to distinguish "commands" (necessary, identifies operation) and "options" (optional, modifies existing operation), we can use the names command
vs option
. A command
can allow multiple options
, and a command can contain smaller commands too.
For example:
git checkout -b <branch>
, checkout
is a command, -b
is an option, <branch>
is a command argument.
rvc edit vpn1 --auto-connect enable
, edit
is command, vpn1
is command argument, --auto-connect
is an option, enable
is the option assignment for --auto-connect
cp /tmp/1 /tmp2
, both /tmp1/
and /tmp2/
are command arguments
aws s3 ls <bucket-name> [options]
, s3
is command, ls
is a command within s3
, <bucket-name>
is argument for ls
command, [options]
are options for the ls
command.
program { name "<program name>" version "<program version>" copyright "<copyright message>" homepage "<homepage URL>" license "<license URL>" }
Let's also add license
!
Options and commands are separate entities. (Sub)commands don't have leading dashes. @ronaldtse called them namespaces - Ie git rm
is rm
in the context of git
, docker rm
is rm
in the context of docker
Suboptions are different. iptables
, for example, has many options that are dependent on specific modules and only available if that module is specified.
iptables -A INPUT -i eth1 -s 10.0.0.0/8 -m limit --limit 5/m --limit-burst 7 -j LOG --log-prefix "LOG_A_FEW: "
iptables -A INPUT -m mac --mac-source 00:0F:EA:91:04:08 -j DROP
Whether we need either of these is undecided. Subcommands are already used by RVC so are probably necessary. Suboptions is less clear. I can see we might use something like
server -p 192.168.0.100:80 --max-connections 24 -p 127.0.0.1:80 --max-connections 4
Multiple args and repeatable options?
format --papersize a4 a5 legal // multiple args
format --papersize a4,a5,legal // one arg
format --papersize a4 --papersize a5 --papersize legal
There's plenty to think about! We need to determine exactly what commandline processing we want available for ongoing/future Ribose products. If we get this right we'll have a suite of products with ergonomic and uniform user interfaces. If we're too restrictive we'll struggle to get command lines to invoke the behaviour we require. And if we're too permissive we'll end up with an unwieldy library that's hard to use and maintain.
Since NOS configuration is compiled with the main program, I suggest the following another section:
program { name "<program name>" version "<program version>" copyright "<copyright message>" homepage "<homepage URL>" }
version, copyright and homepage may be used for command line to show version info.
This is a good idea @jjr840430. Maybe not
name
as it's generallyargv[0]
. Unless there is any occasion when you'd want to report any name other than the name used to invoke the program itself?
@drystone I think we may add API function to update program
info.
I think it's not clear between command
and option
.
Here is an example:
./example -c test
./example -create test
The example
utility has -c
or -create
to create something with a name test
.
Those have the same behavior and output.
But someone could think -c
as option
, -create
as command
.
I think we need to use a popular term option
in NOS syntax.
It will work for all command line formats without any collisions.
If we need to separate them to command
, subcommand
, option
, suboption
, then we need to declare own command line formats and let programmers write NOS syntax based on them.
so I think we may assume only option
and suboption
.
option
and suboption
is statically defined option strings by the programmer, and the remainings which are given at runtime are arguments
.
In the above example, -c
and -create
are options
which was statically defined by the programmer, and test
is argument
which was given at run-time.
Agreed. Program name can be user facing, eg. Riffol
. Binary name is what is used to invoke and is argv[0]
(with path component removed) eg. riffol
-create
!= --create
Long options have the form --option
. Some programs ignore this and it's irritating :)
There is a potential problem though -create
may be parsed as -c reate
-c -r eate...
-c -r -e -a -t -e`. I guess option names should take precedence. We'll be using option parsing libraries (such as getopt) so we'll have to rely on their behaviour in these cases.
If we need to separate them to command, subcommand, option, suboption, then we need to declare own command line formats and let programmers write NOS syntax based on them.
so I think we may assume only option and suboption. option and suboption is statically defined option strings by the programmer, and the remainings which are given at runtime are arguments.
command
is implicit - it's the program itself. How would someone implement rvc edit ...
without some form of subcommand
?
rvc --edit test --auto-connect disable
We may think edit
as option
, --auto-connect
as suboption
.
And test
and disable
is argument
for them.
Also, some command line utilities may not support getopt
or getopt_long
supported by libc
.
So we may think about -create
option.
It means that we don't need to rely on getopt
or getopt_long
standard functions.
For example, macOS has networksetup
utility which has the following example:
networksetup -listnetworkserviceorder
My opinion what could be improved on command line args config, probably this could be helpful in some way:
option encrypt "-e", "--encrypt", "encrypt" {
usage "Encrypt file"
}
option encrypt "-e", "--encrypt", "encrypt" {
type command
}
option sym-algorithm "aes128", "aes256", "3des" {
type switch
default "aes128"
}
option userid "-u", "--userid" {
type string
}
option encrypt "-e", "--encrypt", "encrypt" {
type command, global
usage "Encrypt file"
suboptions {
sym-algorithm?,
userid+,
source+,
destination?
}
}
option decrypt "-d", "--decrypt", "decrypt" { type command, global usage "Decrypt file" suboptions { password?, source+, destination? } }
option password "-p", "--password" { type string, global }
option source { type path default stdin }
option destination { type path default stdout }
Symbols like "?", "+", "!" or keywords could be used to set whether further params are required/variable/can be stacked.
Thanks @ni4 .
If I understand correctly, you're advocating subcommands but as an option subtype. I think they're sufficiently different to have separate names. A subcommand is a set of options and an option cannot contain other options. If a subcommand is selected on a command line, all other subcommands (and their associated options) are ignored. Normal options aren't mutually exclusive in this way.
Suboptions, however, can be described using the option constraints you suggested. Perhaps requires
and conflicts
would be sufficient?
option global { ... }
command cmd {
option cmdopt1 {
....
conflicts [cmdopt2]
}
option cmdopt1_1 {
...
requires [cmdopt1]
}
option cmdopt2 {
....
conflicts [cmdopt1]
}
command nested {
option nestedopt { ... }
}
}
option encrypt "-e", "--encrypt", "encrypt" { ... }
looks tidy but would be parsed into nested dicts that would need flattening:
option encrypt {
"-e" {
"--encrypt" {
"encrypt" { ... }
}
}
}
... /can be stacked
Could you explain what 'stacked' means here?
Sorry for the late reply, but I think by "stacked" @ni4 meant that an option can be used with another option. I think the graph sort of syntax (conflicts [opt]
, requires [opt]
) could be extended (e.g., allows [opt]
) to indicate which options are compatible with what other options.
Is this correct @ni4 ?
@ronaldtse @drystone Sorry for a late response, missed notification. Yeah, @ronaldtse is right - so options may have other suboptions, and so on, without limitation of nesting levels. So we'll have tree, where root is command line, first level - commands/global options, and each of them may have subcomands/suboptions.
Also idea is to describe suboptions/subcommands just by their name to allow to reuse them - say, hash algorithm param with predefined list of constants may be reused for key generation, signing, whatever else. I.e. instead of
command cmd {
option cmdopt1 {
.... option description...
}
}
have
command cmd {
option cmdopt1 required;
}
option cmdopt1 {
.... option description...
}
@ronaldtse @drystone Do you have suggestions related to usage syntax from NOS?
@jjr840430 do you mean the syntax of the usage
strings for individual options?
@ronaldtse @drystone
I would like to discuss about NOS and NOC syntax using the example. Here is the NOS configuration example what
rvc
is using:Here is NOC configuration for
rvc
.