Confutils is a library that aims to solve the configuration problem with a holistic approach. The run-time configuration of a program is described as a plain Nim object type from which the library automatically derives the code for handling command-line options, configuration files and other platform-specific sources such as the Windows registry.
The library focuses on providing a lot of compile-time configurability and extensibility with a strong adherence to the DRY principle.
Let's illustrate the API with a highly annotated example. Our configuration might be described in a separate module looking like this:
# config.nim
import
confutils/defs
type
NimbusConf* = object
#
# This is our configuration type.
#
# Each field will be considered a configuration option that may appear
# on the command-line, whitin an environment variable or a configuration
# file, or elsewhere. Custom pragmas are used to annotate the fields with
# additional metadata that is used to augment the behavior of the library.
#
logLevel* {.
defaultValue: LogLevel.INFO
desc: "Sets the log level" }: LogLevel
#
# This program uses a CLI interface with sub-commands (similar to git).
#
# The `StartUpCommand` enum provides the list of available sub-commands,
# but since we are specifying a default value of `noCommand`, the user
# can also launch the program without entering any particular command.
# The default command will also be omitted from help messages.
#
# Please note that the `logLevel` option above will be shared by all
# sub-commands. The rest of the nested options will be relevant only
# when the designated sub-command is being invoked.
#
case cmd* {.
command
defaultValue: noCommand }: StartUpCommand
of noCommand:
dataDir* {.
defaultValue: getConfigDir() / "nimbus"
desc: "The directory where nimbus will store all blockchain data."
abbr: "d" }: DirPath
bootstrapNodes* {.
desc: "Specifies one or more bootstrap nodes to use when connecting to the network."
abbr: "b"
name: "bootstrap-node" }: seq[string]
bootstrapNodesFile* {.
defaultValue: ""
desc: "Specifies a line-delimited file of bootsrap Ethereum network addresses"
abbr: "f" }: InputFile
tcpPort* {.
desc: "TCP listening port" }: int
udpPort* {.
desc: "UDP listening port" }: int
validators* {.
required
desc: "A path to a pair of public and private keys for a validator. " &
"Nimbus will automatically add the extensions .privkey and .pubkey."
abbr: "v"
name: "validator" }: seq[PrivateValidatorData]
stateSnapshot* {.
desc: "Json file specifying a recent state snapshot"
abbr: "s" }: Option[BeaconState]
of createChain:
chainStartupData* {.
desc: ""
abbr: "c" }: ChainStartupData
outputStateFile* {.
desc: "Output file where to write the initial state snapshot"
name: "out"
abbr: "o" }: OutFilePath
StartUpCommand* = enum
noCommand
createChain
#
# The configuration can use user-defined types that feature custom
# command-line parsing and serialization routines.
#
PrivateValidatorData* = object
privKey*: ValidatorPrivKey
randao*: Randao
Then from our main module, we just need to call confutils.load
which must be
given our configuration type as a parameter:
# main.nim
import
confutils, config
when isMainModule:
let conf = NimbusConf.load()
initDatabase conf.dataDir
And that's it - calling load
with default parameters will first process any
command-line options and then it will
try to load any missing options from the most appropriate
configuration location
for the platform. Diagnostic messages will be provided for many simple
configuration errors and the following help message will be produced
automatically when calling the program with program --help
:
Usage: beacon_node [OPTIONS] <command>
The following options are supported:
--logLevel=LogLevel : Sets the log level
--dataDir=DirPath : The directory where nimbus will store all blockchain data.
--bootstrapNode=seq[string] : Specifies one or more bootstrap nodes to use when connecting to the network.
--bootstrapNodesFile=FilePath : Specifies a line-delimited file of bootsrap Ethereum network addresses
--tcpPort=int : TCP listening port
--udpPort=int : UDP listening port
--validator=seq[PrivateValidatorData] : A path to a pair of public and private keys for a validator. Nimbus will automatically add the extensions .privkey and .pubkey.
--stateSnapshot=Option[BeaconState] : Json file specifying a recent state snapshot
Available sub-commands:
beacon_node createChain
--out=OutFilePath : Output file where to write the initial state snapshot
For simpler CLI utilities, Confutils also provides the following convenience APIs:
import
confutils
cli do (validators {.
desc: "number of validators"
abbr: "v" }: int,
outputDir {.
desc: "output dir to store the generated files"
abbr: "o" }: OutPath,
startupDelay {.
desc: "delay in seconds before starting the simulation" } = 0):
if validators < 64:
echo "The number of validators must be greater than EPOCH_LENGTH (64)"
quit(1)
import
confutils
proc main(foo: string, bar: int) =
...
dispatch(main)
Under the hood, using these APIs will result in calling load
on an anonymous
configuration type having the same fields as the supplied proc params.
Any additional arguments given as cli(args) do ...
and dispatch(fn, args)
will be passed to load
without modification. Please note that this requires
all parameters types to be concrete (non-generic).
This covers the basic usage of the library and the rest of the documentation will describe the various ways the default behavior can be tweaked or extended.
A number of pragmas defined in confutils/defs
can be attached to the
configuration fields to control the behavior of the library.
template desc*(v: string) {.pragma.}
A description of the configuration option that will appear in the produced help messages.
template longDesc*(v: string) {.pragma.}
A long description text that will appear below regular desc. You can use one of {'\n', '\r'} to break it into multiple lines. But you can't use '\p' as line break.
-x, --name regular description [=defVal].
longdesc line one.
longdesc line two.
longdesc line three.
template name*(v: string) {.pragma.}
A long name of the option.
Typically, it will have to be be specified as --longOptionName value
.
See Handling of command-line options
for more details.
template abbr*(v: string) {.pragma.}
A short name of the option.
Typically, it will be required to be specified as -x value
.
See Handling of command-line options
for more details.
template defaultValue*(v: untyped) {.pragma.}
The default value of the option if no value was supplied by the user.
template required* {.pragma.}
By default, all options without default values are considered required.
An exception to this rule are all seq[T]
or Option[T]
options for
which the "empty" value can be considered a reasonable default. You can
also extend this behavior to other user-defined types by providing the
following overloads:
template hasDefault*(T: type Foo): bool = true
template default*(T: type Foo): Foo = Foo(...)
The required
pragma can be applied to fields having such defaultable
types to make them required.
template command* {.pragma.}
This must be applied to an enum field that represents a possible sub-command. See the section on sub-commands for more details.
template argument* {.pragma.}
This field represents an argument to the program. If the program expects
multiple arguments, this pragma can be applied to multiple fields or to
a single seq[T]
field depending on the desired behavior.
template separator(v: string)* {.pragma.}
Using this pragma, a customizable separator text will be displayed just before this field. E.g.:
Network Options: # this is a separator
-a, --opt1 desc
-b, --opt2 desc
---------------- # this is a separator too
-c, --opt3 desc
The confutils/defs
module provides a number of types frequently used
for configuration purposes:
InputFile
, InputDir
Confutils will validate that the file/directory exists and that it can be read by the current user.
ConfigFilePath[Format]
A file system path pointing to a configuration file in the specific format.
The actual configuration can be loaded by calling load(path, ConfigType)
.
When the format is WindowsRegistry
the path should indicate a registry key.
OutPath
A valid path must be given.
Furthermore, you can extend the behavior of the library by providing overloads such as:
proc parseCmdArg*(T: type Foo, p: string): T =
## This provides parsing and validation for fields having the `Foo` type.
## You should raise `ConfigurationError` in case of detected problems.
...
proc humaneTypeName*[T](_: type MyList[T]): string =
## The returned string will be used in the help messages produced by the
## library to describe the expected type of the configuration option.
mixin humaneTypeName
return "list of " & humaneTypeName(T)
For config files, Confutils can work with any format supported by the
nim-serialization library
and it will use the standard serialization routines defined for the field
types in this format. Fields marked with the command
or argument
pragmas
will be ignored.
Confutils includes parsers that can mimic several traditional styles of
command line interfaces. You can select the parser being used by specifying
the CmdParser
option when calling the configuration loading APIs.
The default parser of Confutils is called MixedCmdParser
. It tries to follow
the robustness principle
by recognizing as many styles of passing command-line switches as possible.
A prefix of --
is used to indicate a long option name, while the -
prefix
uses the short option name. Multiple short options such as -a
, -b
and
-c
can be combined into a single -abc
string. Both the long and the short
forms can also be prefixed with /
in the style of Windows utilities. The
option names are matched in case-insensitive fashion and certain characters
such as _
and -
will be ignored. The values can be separated from the
option names with a space, colon or an equal sign. bool
flags default to
false
and merely including them in the command line sets them to true
.
Other provided choices are UnixCmdParser
, WindowsCmdParser
and NimCmdParser
which are based on more strict grammars following the most established
tradition of the respective platforms. All of the discussed parsers are
defined in terms of the lower-level parametric type CustomCmdParser
that
can be tweaked further for a more custom behavior.
Please note that the choice of CmdParser
will also affect the formatting
of the help messages. Please see the definition of the standard Windows
or Posix command-line help syntax for mode details.
As seen in the introduction example, Confutils makes it
easy to create command-line interfaces featuring sub-commands in the style
of git
or nimble
. The structure of the sub-command tree is encoded as
a Nim case object where the sub-command name is represented by an enum
field having the command
pragma. Any nested fields will be considered
options of the particular sub-command. The top-level fields will be shared
between all sub-commands.
For each available choice of command and options, Confutils will automatically
provide a help
command and the following additional switches:
-h
will print a short syntax reminder for the command--help
will print a full help message (just like the help
command)After parsing the command line options, the default behavior of Confutils is
to try to fill any missing options by examining the contents of the environment
variables plus two per-user and system-wide configuration locations derived from
the program name. If you want to use Confutils only as a command-line processor
or a config file parser for example, you can supply an empty/nil value to the
cmdLine
, envTable
or configFileEnumerator
parameters of the load
call.
More specifically, the load
call supports the following parameters:
cmdLine
, envTable
The command-line parameters and the environment table of the program.
By default, these will be obtained through Nim's os
module.
EnvValuesFormat
, envVarsPrefix
A nim-serialization format used to deserialize the values of environment
variables. The default format is called CmdLineFormat
and it uses the
same parseCmdArg
calls responsible for parsing the command-line.
The names of the environment variables are prefixed by the name of the
program by default and joined with the name of command line option, which is
uppercased and characters -
and spaces are replaced with underscore:
let env_variable_name = &"{prefix}_{key}".toUpperAscii.multiReplace(("-", "_"), (" ", "_"))
configFileEnumerator
A function responsible for returning a sequence of ConfigFilePath
objects.
To support heterogenous config file types, you can also return a tuple of
sequences. The default behavior of Windows is to obtain the configuration
from the Windows registry by looking at the following keys:
HKEY_CURRENT_USER/SOFTWARE/{appVendor}/{appName}/
HKEY_LOCAL_MACHINE/SOFTWARE/{appVendor}/{appName}/
On Posix systems, the default behavior is attempt to load the configuration from the following files:
/$HOME/.config/{appName}.{ConfigFileFormat.extension}
/etc/{appName}.{ConfigFileForamt.extension}
ConfigFileFormat
A nim-serialization format that will be used by default by Confutils.
The load
call offers few more optional parameters for modifying the
produced help messages:
bannerBeforeHelp
A copyright banner or a similar message that will appear before the automatically generated help messages.
bannerAfterHelp
A copyright banner or a similar message that will appear after the automatically generated help messages.
version
If you provide this parameter, Confutils will automatically respond
to the standard --version
switch. If sub-commands are used, an
additional version
top-level command will be inserted as well.
confutils_colors
This option controls the use of colors appearing in the help messages produced by Confutils. Possible values are:
NativeColors
(used by default)
In this mode, Windows builds will produce output suitable for the console
application in older versions of Windows. On Unix-like systems, this is
equivalent to specifying AnsiColors
.
AnsiColors
Output suitable for terminals supporting the standard ANSI escape codes: https://en.wikipedia.org/wiki/ANSI_escape_code
This includes most terminal emulators on modern Unix-like systems, Windows console replacements such as ConEmu, and the native Console and PowerShell applications on Windows 10.
None
or NoColors
All output will be colorless.
The development of Confutils is sponsored by Status.im through the use of GitCoin. Please take a look at our tracker for any issues having the bounty tag.
When submitting pull requests, please add test cases for any new features
or fixes and make sure nimble test
is still able to execute the entire
test suite successfully.
Licensed and distributed under either of
or
at your option. This file may not be copied, modified, or distributed except according to those terms.