Open epage opened 2 years ago
Comment by kbknapp Monday Nov 14, 2016 at 21:30 GMT
This relates to #251
I'm not against this at all, but I don't have the bandwidth to implement it just yet. I'll keep it on the issue tracker. I'd say supporting YAML and TOML would be fine.
The part I'd still need fleshed out is whether a user can use both the command line and this config file as well? Does one override the other? What about arguments that accept multiple values, and are passed both in the command line and config files, do they get overridden or appended to?
Comment by casey Tuesday Nov 15, 2016 at 05:19 GMT
I think that a MergeType
enum would work, something like:
enum MergeType {
Override, // replace values with those from config file
Defaults, // use values from config file only where not given on command line
Conflict. // complain if any value is provided in both places
}
...
matches.merge_from_config(config, MergeType::Override);
For arguments that accept multiple values, I think that until someone has a specific use case for merging values from both places, it's probably best to only allow overriding or ignoring.
Comment by ssokolow Wednesday Nov 23, 2016 at 22:56 GMT
This is actually something I'm going to have to deal with in a project soon. (In Python, I tend to dodge the issue by hard-coding my defaults at the top of the only (or top-level) .py
file and telling users to just edit them there).
While I'm not familiar with clap internals, I don't see why it would be necessary to over-complicate things. Let's start by separating what clap needs out from a more generic "config file for my program" solution.
--append=this --append=that
.fancy_cp <file> [...] [target_dir]
would be an example where it'd actually make sense but be esoteric.)MergeType
is worth the trouble. Generally, a much better solution is to just use simple linear precedence:
Override
wouldn't be frustrating and/or confusing is when a --foo-from-file=
argument comes after a --foo=
argument in the command-line... so just add "all from file" and "specific field from file" as argument types and call it a day. (ie. --foo=A --foo=B_trumps_A --foo-from-file=contents_trump_B --foo=winner
.)Conflict
making sense is within the same source (ie. Complaining about specifying two values within the same config file or on the same command line.), so supporting it in an intuitive way is orthogonal to supporting config files. (The key being to understand what purpose Conflict
actually serves. Forcing people to modify config files for a one-off run is never a good design (it encourages human error), so Conflict
only really makes practical sense as a way to say that a single source is internally contradictory.)With all of that said, what I'd suggest is that it's not really the config file parsing you need to focus on, but providing a clearly documented API that allows something like (human_readable_source_identifier, parsed_data_structure)
to be fed into clap so we can feed in JSON/TOML/whatever at our leisure. (The human-readable source identifier would be used for error messages.)
That would also allow clap defaults to be embedded into larger JSON/TOML/etc. configuration files without complicating clap.
Heck, with a little thought, the "from command-line arguments" functions you already have could just be parsers on the same tier as JSON/TOML/etc. which just feed into the API I'm proposing.
Comment by BurntSushi Thursday Nov 24, 2016 at 03:59 GMT
The only situation where I've seen Conflict making sense is within the same source (ie. Complaining about specifying two values within the same config file or on the same command line.), so supporting it in an intuitive way is orthogonal to supporting config files.
I'm not sure I really buy this. If I have a flag --foo that takes a value N, then it seems reasonable for me to want to assign a default value to it in a config file, but retain the right to want to override it on the actual command line. Depending on how the --foo flag was defined, this might not be allowed in the form of --foo X --foo Y
, so a config file might need special handling?
Comment by ssokolow Thursday Nov 24, 2016 at 05:36 GMT
I'm not sure I really buy this. If I have a flag --foo that takes a value N, then it seems reasonable for me to want to assign a default value to it in a config file, but retain the right to want to override it on the actual command line. Depending on how the --foo flag was defined, this might not be allowed in the form of --foo X --foo Y, so a config file might need special handling?
You misunderstand. I made only two claims and neither was that Conflict
should be mandatory within a single source. Those two claims are:
Conflict
never produces a worthwhile behaviour when applied to two pieces of data from different sources.MergeType
behaviour is handled, it is entirely internal to the parsing of each individual source.What I'm arguing is that:
MergeType
now, there's nothing about adding support for configuration files which adds it to the list of requirements.MergeType
urgently and implement it before config file support, no beneficial functionality can be gained by expanding that "within a single source" implementation to also operate between multiple sources.Hence, orthogonal.
First, let's discuss the "never produces worthwhile behaviour" point, starting with a few examples:
~/.user_default
can't override /etc/global_default
, then that's bad.--one-time-setting
can't override either of those, then that's bad.~/.defaults_a
and ~/.defaults_b
don't have a defined precedence order, you're forcing users to manage pointless busywork in the best case and, in the worst case, you wind up with a "FAT table copies 1 and 2 disagree? Which one did you intend?" situation.This derives from two issues:
Or, let's come at it from the other direction and address Override
, Defaults
, and Conflict
as options.
Defaults
needs no justification. When applied to reconcile data from two different sources, it implements the behaviour configuration files serve in every case I've ever encountered.
Conflict
is the hardest to justify because:
--foo=false
unless you call it with --foo=true
.)/etc/config_defaults
specified foo=false
".The former case is counter-productive and the latter sounds like trying to kitchen-sink some kind of user permissions system into clap when that's a job for code which runs after clap is done.
Finally, Override
is just Defaults
with the ordering swapped, so it's only justifiable in the one case where you can't swap the order: The command-line arguments, which must always be the last in the chain.
The problem is that, by their very nature, command-line arguments are also the most suitable for one-off overrides, which means that there is no justifiable reason to use Override
to silently ignore what the user specified on the command-line because some config file somewhere disagrees.
Therefore, it's much better to just make defined-precedence overriding an inherent, unavoidable feature of how conflicts between multiple sources are resolved. It's the de facto standard way to handle things and it's the only solution which addresses all of the concerns I brought up.
...it's also easy for people to conceptualize. For example, in Python (since it's pseudocode-like):
matches = dict()
matches.update(parse_global_config())
matches.update(parse_local_config())
matches.update(parse_command_line_args())
Therefore, the concept of MergeType
is only meaningful within the process of parsing a single source, because merging input from multiple sources should always Defaults
.
We may indeed need to discuss the MergeType
idea more, but the conclusion we come to has no bearing on a well-designed multi-source implementation.
--config-file
or the like)The only potential kink I see is to at least warn users that step 3 might reset config_file
so it's not the actual value used if the config file itself contains a config_file
value.
The beauty of this approach is that it's made of very small, very extensible pieces and you can get a lot of utility early on, then amend it later. Here are some example steps:
argv
parsing out from the rest so that anyone can implement the connecting interface on top of a JSON/TOML/etc. parser. (Now, even if they have to manually hook up the config parser and manually merge things, users don't have to reinvent clap's "apply the validation" code in between those two steps.)What I'm envisioning is that the argv
-parsing side of the interface would take a reference it could use to query the schema-validation side to disambiguate things that would be inherent in the framing of some formats but not others. For example:
argv
can resolve command-line shorthands like -s
without complaint while config files can print a deprecation message or exit with a failure message asking the user if they meant sidechannel_data=...
in config.cfg
)argv
deciding whether --foo --bar
actually means --foo=--bar
)argv
can use --foo=this --foo=that --foo=those
while the internal representation it outputs can be closer to the {"foo": ["this", "that", "those"]}
that JSON would use.)Comment by BurntSushi Thursday Nov 24, 2016 at 22:51 GMT
@ssokolow Frankly, I'm having a really hard time following any of what you're saying. In particular, I find it hard to tie what you're saying to a concrete user experience. I'm probably missing some important context. In any case, I discussed some of the issues I personally see with config files: https://github.com/BurntSushi/ripgrep/issues/196#issuecomment-262853109
Comment by ssokolow Friday Nov 25, 2016 at 00:41 GMT
Ugh. That's the worst kind of response because it's a perfectly fair one, yet it's the hardest to formulate a meaningful answer to. Give me an hour or two to do "morning routine" stuff and I'll see what I can do to come at it from a different angle and simplify it so that, at the very least, maybe we can figure out where the disconnect is.
Comment by ssokolow Friday Nov 25, 2016 at 01:29 GMT
OK. From a user-experience standpoint, the most intuitive solution is to have later arguments always override earlier ones and to treat the fallback chain from command-line to user defaults to global defaults the same way.
That gets you 99% of the way to supporting all behaviours I've seen in the wild.
For example:
/etc/global_defaults
could set foo=bar
~/.user_defaults
could override it with foo=baz
alias mycmd="mycmd --foo=quux"
could override it with foo=quux
mycmd --foo=glorp
could resolve to mycmd --foo=quux --foo=glorp
The end result clap should return is foo=glorp
in that circumstance, rather than erroring out because some prior default the user had forgotten about is in conflict with what they want to do this one time.
(ie. They shouldn't have to type command mycmd --foo=glorp
to explicitly bypass the shell alias
or edit configuration files and then edit them back.)
What I'm arguing is that it's very simple to accomplish this as a collection of small, easily-composed pieces:
--foo=bar --foo=baz
resolution mentioned above for non-appending arguments into the default. (Having it error out is, at best, useful only in niche situations thanks to the alias
shell built-in and wrapper scripts which don't carry around their own argument parsers.)get_matches
and friends into two pieces:
argv
string to a structured internal representation. (With access to query the schema to disambiguate things like "Does --foo
take a value?" and "Does ---bar
append, count, or override when specified multiple times?".)argv
now produces.get_matches()
) and merges them in a "last in the list wins" manner.Getting paths to things like XDG configuration directories is external to clap, because we don't want to preclude parsing config files stored elsewhere or tie clap to a specific implementation.
Comment by kbknapp Friday Nov 25, 2016 at 02:21 GMT
I've read all the comments and have some thoughts on the matter but I'm on mobile right now so I'll try to write up my thoughts early to tomorrow. It boils down to I'd like command line to override config files or env vars, but those two extra sources to be added in a first come first serve manner. I'd also like to limit claps responsibility to parsing arguments from a source. Not determining source order or source validity.
I'll type up my full thoughts soon.
Comment by ssokolow Friday Nov 25, 2016 at 02:37 GMT
It boils down to I'd like command line to override config files or env vars, but those two extra sources to be added in a first come first serve manner. I'd also like to limit claps responsibility to parsing arguments from a source. Not determining source order or source validity.
I fully agree. Hence my idea limiting clap to:
argv
to structured representation (more akin to what JSON can represent)It then becomes easy for the clap-using application or 3rd-party crates to implement things like:
shlex
to tokenize an environment variable's contents into what get_matches_from
expects.As long as you provide examples of how to accomplish all of this with minimal boilerplate using 3rd-party crates, you can indefinitely defer the question of whether anything else belongs in clap itself.
I'll type up my full thoughts soon.
I look forward to it.
Comment by kbknapp Saturday Nov 26, 2016 at 21:58 GMT
Ok so it took me a little longer to get to this than I'd planned, but here's where I stand on the issue.
First, I'm not terribly interested in clap handling the file I/O part of this, i.e. the reading files, handling permissions, errors that come from this, precedence, etc. I intend for this to all happen in the consumer code. The consumer would then pass in the deserialized config representation, for now I'm only imagining TOML/YAML, but others could be added.
The goal is ultimately to get clap the info it needs to do it's job, i.e. a normalized structure of an argv. In clap's case simply an ordered vector of strings (meaning anything from OsStr, String, etc.). I'm OK with giving clap either a Toml
or Yaml
object, and having clap normalize it down to just the ordered Vec
.
The details would probably be something like defining a trait Normalize
(aside, all names in this proposal are 🚲 🏠-able) that does the normalizing, and as a start implementing that trait on Yaml
, Toml
, String
, and OsString
(the latter two in order to support env vars). The App
struct would then accept any number of "external argv" sources via something like fn external_argv<T: Normalize>(argv: T)
, and save these in a first come first serve basis. I.e. the pseudocode below is the same to clap, and totally up to the consumer to decide (which allows us to side step all this precedence discussion and allow clap to stay more focused on a single purpose):
let some_env_var = env::var("SOME_ENV_VAR").ok();
let global_cfg = load_toml("/etc/myconfig.toml");
let user_cfg = load_toml("~/myconfig.toml");
let m = App::new("test")
.external_argv(some_env_var)
.external_argv(global_cfg)
.external_argv(user_cfg)
.get_matches();
If the consumer wants global_cfg
to take precedence over user_cfg
for whatever reason, they'd swap those two lines.
This makes parsing very simple because internally clap just parses them in reverse order, and if it reaches an arg that arleady exists in the matches, it just skips it.
There are two outstanding issues that this would present though. One is if you have an arg that accepts multiple values, and has a value in some external argv, should a later value in either another external argv or via the explicit command line add to these values, or entirely override? The proposed system above entirely overrides, which I'm more of a fan of. If a consumer wants to provide a global default and allow users to add to those values, I'd be easiest to simply tell the user to include that default value in their own "overrides" or just re-add that value back after clap is done with it's parsing.
The second issue where this MergeType came into play. Since clap allows two types of conflicts, POSIX style overrides and hard conflicts, a MergeType would allow consumers to effectively convert from hard conflicts to POSIX style overrides only in the case of external argv conflicts. Basically it gives the choice of whether they want hard conflicts or overrides.
This situation, in my estimation, only happens in user defined configs. I.e. a user wants to specify a default that conflicts with other options, but yet may want to override that behaviour at some point. I can't imagine why I consumer would put a conflicting argument into a default config and actually want a hard conflict. Think of unix style aliases, ls="ls -l"
yet due to POSIX style overrides, using values that conflict with -l
is perfectly fine, so long as they come afterwards. However, as the developer/consumer it's sometimes difficult to decide what should be a hard conflict and what should be POSIX overridable because overrides sometimes seem confusing at runtime, whereas with a hard conflict, the user knows exactly what is going on and how to fix it.
I'm of the thought that all conflicts arising from external argvs should be treated as overridable, and it should just be documented well. I can't think of a concrete example where I'd actually want a hard conflict because I explicitly set something via the commandline.
This may sound like I'm in favor of a MergeType, but actually the more I think about it, the less I am. As I stated earlier, I'd prefer to treat all things as overridable, and disallow adding values at the commandline to values defined in the configs.
The only thing left to determine is how to represent free/positional arguments in these configs. Another 🚲 🏠 for sure, but options, and flags are simple. I'd suggest simply using a single "args" key and assigning the values in sequence such as args="foo bar baz"
.
Thoughts?
Comment by kbknapp Saturday Nov 26, 2016 at 22:39 GMT
I wasn't clear about the positional args part, we could equally as easy use the key to individualize them, but I kind of like that they're forced to be in order with only a single key to keep from any confusion by accidentally putting them in the wrong order
Comment by ssokolow Sunday Nov 27, 2016 at 05:04 GMT
We basically agree on the design aside from whether the merging should be declarative or procedural.
Your declarative approach is definitely nicer to look at, but it's puts more onus on clap to support edge-case features (or constrain users by refusing to), as I'll answer in reply to one of your outstanding issues...
There are two outstanding issues that this would present though. One is if you have an arg that accepts multiple values, and has a value in some external argv, should a later value in either another external argv or via the explicit command line add to these values, or entirely override? The proposed system above entirely overrides, which I'm more of a fan of. If a consumer wants to provide a global default and allow users to add to those values, I'd be easiest to simply tell the user to include that default value in their own "overrides" or just re-add that value back after clap is done with it's parsing.
That's part of the reason I wanted the merge to be a later step. It allows these two cases to be implemented in the consumer:
Supporting things like --config-file
. With your solution, one of the following has to happen:
a. Clap needs explicit support for a new argument type
b. Users need to call get_matches
, then external_argv(load_toml(matches.value_of('config_file').unwrap()))
, then get_matches
again.
With my solution, they just call get_matches
, then call something in the vein of get_matches_via_normalize(load_toml(matches.value_of('config_file').unwrap()))
, then call a clap-provided merging function like matches1.update(matches2)
.
No special "look ahead, then dynamically inject an external_arg
" or "re-parse from the beginning with changed parameters" necessary.
Allowing users to control behaviour of multiple arguments.
With your solution, clap dictates how it works. With my solution, users can easily extract the values which should add together before doing the merging and then add them together manually.
It also has the benefit that there's less uncertainty about whether clap will allow users to reuse the same App
to parse multiple sets of merged inputs. That is, I'd be worried that external_argv
might invalidate earlier references, requiring me to throw out App
and create it anew every time when I need to do something like this:
let m = App::new("test")
let matches1 = m
.external_argv(some_env_var1)
.get_matches_from(args1);
let matches2 = m
.external_argv(some_env_var2)
.external_argv(user_cfg2)
.get_matches_from(args2);
let matches3 = m
.external_argv(some_env_var3)
.external_argv(global_cfg3)
.external_argv(user_cfg3)
.get_matches_from(&[]);
The only thing left to determine is how to represent free/positional arguments in these configs. Another :bike: :house: for sure, but options, and flags are simple. I'd suggest simply using a single "args" key and assigning the values in sequence such as args="foo bar baz".
I wasn't clear about the positional args part, we could equally as easy use the key to individualize them, but I kind of like that they're forced to be in order with only a single key to keep from any confusion by accidentally putting them in the wrong order
At the very least, you'll want it to be args=["foo", "bar", "baz"]
to avoid reinventing quoting/escaping.
With that said, this is definitely a tricky thing to address because:
Mapping one key to multiple schema entries feels like undesirable magic behaviour and makes external_argv
more than merely a merging, trait-enabled version of get_matches_from
, which also feels conceptually wrong.
If clap doesn't enforce the "list of positional arguments is never sparse" invariant, mapping them to individual keys in config files could introduce potential for logic bugs.
If clap does enforce non-sparseness for positional arguments, it could frustrate users when cmd foo bar
keeps turning into "cmd foo bar baz` yet the shell is adamant that it's not meddling with it.
(Wrapper scripts and shell functions are the standard, accepted way to augment or meddle with positional arguments. As someone who cares about user experience, I might go as far as slipping my own filter in between load_toml
and clap if it didn't provide a way to opt out of positional arguments into the config file... and then we're right back to reinventing the config file schema because clap's implementation is too inflexible.)
I'd just treat positional arguments in config files as a validation failure and wait to see if anyone complains, since it can be relaxed without breaking anything. (I've never seen a config file or environment variable that allows specifying positional arguments in 10 years of using DOS/Windows command-line applications and another 16 of using Linux ones. It's always been wrapper scripts or shell features like alias
and function done() { command mv "$@" ~/done/; }
)
...or, in the case of DOS and Windows, wrapper scripts as .BAT
files would looke like these:
REM DOS
move %1 %2 %3 %4 %5 %6 %7 %8 %9 %HOME%\DONE\
REM Windows
move %* %HOME%\Done\
Finally, since you didn't mention them either way, here are a couple of other points:
I have no problem with Normalize
as long as it's implemented such that, if .external_argv
accepts the output of load_toml
directly, there's also an easy way to use only a specified subtree, so the file can contain multiple sections with only one of them being for clap.
You'll need Normalize
implementations to have access to the schema because, otherwise, the only attainable normalized form is to flatten structured data from TOML/JSON/YAML/etc. into an argv
Vec, just so they can be re-structured again once access to the schema is available... and that feels very unpleasant to think about.
(Without access to the schema, tell me whether --foo --bar
is {"foo": true, "bar": true}
or {"foo": "--bar"}
... or {'foo': 1 + (-1)}
for that matter, if you're dealing with options like --verbose
and --quiet
.)
Comment by kbknapp Monday Nov 28, 2016 at 18:14 GMT
Supporting things like --config-file. [...]
IMO, that's a little bit of a niche use-case. I'm not saying it's uncommon, but I think it's FAR more common to simply provide these "defaults" in a predetermined location. Unless you're using aliases, typing $ myprog --config-file foo.toml
is basically the same as providing those defaults in the first place.
Due to how the internals of clap work, you can't just have a matches.update(load_yaml!("foo.toml"))
, at a minimum you'd have to provide the App
instance too because that's where the schema is defined. So it would end up being something long the lines of app.update_matches_from(load_yaml!("foo.yaml"), &mut matches)
Also parsing the arguments is very fast, it's building the App
instance that takes more of the time. So calling get_matches
more than once shouldn't be a real issue in practice, especially if it's only for a small percentage of uses cases (such as --config-file <file>
). I'm not saying this is the best solution, but it's definitely the easiest considering I'll have to support this solution 😉
there's less uncertainty about whether clap will allow users to reuse the same App to parse multiple sets of merged inputs.
There shouldn't be any issue with re-using the same App
instance to update the matches. This may just need to be documented better in this case. As for the example with matches{1,2,3}
, it would depend on how you're trying to do this, but I'd need to see a real use case. The example you posted looks like it would work via this app.update_matches_from()
, but not work with an app.external_argv()
. Because the app.external_argv()
would always allow adding to the App
instance, but not taking away from it upon additional parses.
have no problem with Normalize as long as it's implemented such that, if .external_argv accepts the output of load_toml directly, there's also an easy way to use only a specified subtree, so the file can contain multiple sections with only one of them being for clap
That's the plan. As far as using a subtree, that should work, depending on the deserialization framework. I.e. this external_argv
would simply take a TomlTable or YamlTree (not reall objects, just abstract ideas), whether thats an entire file's table/tree or just a subset doesn't really matter. If the consumer only wants to provide a subset, they just give that subset to the external_argv
and not the entire deserialized file.
You'll need Normalize implementations to have access to the schema because, otherwise, the only attainable normalized form is to flatten structured data from TOML/JSON/YAML/etc. into an argv Vec, just so they can be re-structured again once access to the schema is available... and that feels very unpleasant to think about. (Without access to the schema, tell me whether --foo --bar is {"foo": true, "bar": true} or {"foo": "--bar"}... or {'foo': 1 + (-1)} for that matter, if you're dealing with options like --verbose and --quiet.)
The App
instance contains the schema. And parsing a flat argv is exactly what clap does already, so that's not an issue at all. In fact, the Normalize
trait would simply flatten the Yaml/Toml/JSON/whatever into what clap is already good at parsing, a flat Vec
.
At the very least, you'll want it to be args=["foo", "bar", "baz"] to avoid reinventing quoting/escaping.
Good point. @nabijaczleweli has a branch with a parser which does all the whitespace/quotation handling that we could use/adapt. Although, the more I think about it, the more I'd rather do one of the following
1="foo", 2="bar"
form where 2
requires 1
...but I still don't really like this because a user that types $ myprog
may not realize what he's really doing is $ myprog foo bar
, but then typing $ myprog baz
does what...$ myprog baz bar
? It just seems like extra magic that could go very wrong. Edit: updated Option 2 about the positional args
Comment by TruputiGalvotas Tuesday Mar 21, 2017 at 11:04 GMT
I don't know whether anyone knows about this here: https://docs.rs/preferences/1.1.0/preferences/
But it seems you could just use this as an optional dependency instead of a separate implementation within clap to achieve the same thing. The clap would still be clap instead of swiss army knife of being a configuration file as well as command line parser.
Comment by kbknapp Monday Feb 05, 2018 at 15:31 GMT
This will be implemented as adding App::argv
which clap will parse prior to the CLI, and the CLI will override options found in any of the App::argv
s.
Comment by lez Monday Mar 12, 2018 at 09:00 GMT
Hi, I would like to work on this issue. Do you see any chance that it can be integrated to the 2.x branch, too, before 3.x comes out?
Comment by kbknapp Monday Mar 19, 2018 at 20:03 GMT
@lez of course I can't stop people from working for free on this project, and all contributions are very welcome! :)
My only caution to people implementing things on 2.x branch is that all of it needs to be "back/forward ported" to 3.x branch. So if the changes are large, it's more difficult to port and thus ends up getting re-implemented anyways and increases the duplication of effort...or just doesn't make it to v3.
With that caveat aside, this particular change if limited to the base case needed should actually be pretty minimal and thus could be directly ported to v3 ;)
The base case of this being:
OsStrings
(as that's what the internal parses iterates over).Imagine we had 2 external argvs, and the command line:
argv1 = ["a", "b", "c"]
argv2 = ["d", "e"]
cli = ["x", "y", "z"]
what_clap_parses = ["a", "b", "c", "d", "e", "x", "y", "z"]
Comment by davidMcneil Tuesday Nov 26, 2019 at 12:38 GMT
This is proof-of-concept work for integrating clap with a config file.
Comment by pksunkara Tuesday Mar 03, 2020 at 13:07 GMT
@CreepySkeleton Please read through this issue in relation to #1693 and how we can combine both of them
Comment by CreepySkeleton Tuesday Mar 03, 2020 at 16:41 GMT
This is not the first time the "external argv" theme pops up, and we have also got few similar requests in structopt: one, two. I have actually read through this issue back in November or so, few rogue thoughts:
We don't want to backport it to 2.x. Manpower is limited and, frankly, scarce. Let's focus on 3.x.
Citing @kbknapp
I'd also like to limit claps responsibility to parsing arguments from a source. Not determining source order or source validity. [...] First, I'm not terribly interested in clap handling the file I/O part of this, i.e. the reading files, handling permissions, errors that come from this, precedence, etc. I intend for this to all happen in the consumer code.
I agree. Otherwise, we risk to stitch yet another kind of Frankenstein's monster, capable of everything but extremely hard to maintain/use.
I like https://github.com/clap-rs/clap/issues/748#issuecomment-263348300 , specifically, the update_matches()
part. I also agree that positional args are exempt from configs (i.e they are allowed only in the last update).
https://github.com/clap-rs/clap/issues/748#issuecomment-374352572 is a simpler option which looks fine too. I vote for it as the bare workable minimum.
I don't think this is related to #1693 . They can coexist.
Comment by Kixunil Tuesday Apr 28, 2020 at 13:33 GMT
If anyone needs this feature more than some other fancies like native subcommand support, you might be interested in my crate configure_me
. From my experience, this is very common if you write long-running services instead of command line tools.
The whole crate took a lot different approach than clap. Mainly it's declarative-first, no stringly manipulation, no forcing into UTF-8 strings. (Disclaimer: maybe clap
has these properties too now or I misunderstood them at the time.) I implemented it as build script instead of proc macro at the time mainly because it was (or seemed to be) simpler. However it turned out to be a better approach anyway. At least for now.
People here raised some interesting issues, so here are my answers - how I solved them in configure_me
:
--arg=val
being same as --arg val
, coalesced flags...)/etc...
, ~/...
), environment variables, arguments--config FILE
argument? The values in that file override appropriate values that were parsed so far, including command-line values. This is consistent, so shouldn't be too surprising.configure_me
can read contents of a directory too - very handy to implement conf.d
- style configurationI think the most important thing about this approach is: it works. Over past years there weren't major complaints about it. Some missing features/bugs were fixed as needed. Feel free to open issues if you find something not working for you.
Comment by kbknapp Tuesday Apr 28, 2020 at 14:08 GMT
I don't think this is related to #1693 . They can coexist.
Agreed.
I think we could combine the two, but I see them as different things. This issue (#748) is about allowing the consumer code to essentially make their own argfile impl if they choose. It's about making it possible to update matches from multiple argv sources (those sources are handled by user code).
I may have already said it elsewhere, but I tend to see this issue being more of just adding either App::argv
which accepts an iterator over OsStrings
(or could be a generic Into<Iterator<Item=OsString>>
), and can be used multiple times (in the builder sense) and all those argv's are concatenated together, and the CLI (std::env::args_os
) is concatenated at the end, and then parsing happens like normal (minus perhaps the exception that instead of conflicts, we use overrides).
Comment by kbknapp Tuesday Apr 28, 2020 at 14:10 GMT
Also, just throwing my two cents into the ring; if I had to pick between this issue and #1693 I'd pick this issue. As implementing this issue would allow users who want an argfile to do so in thier own code, but the other way around isn't as easy.
Comment by pksunkara Tuesday Apr 28, 2020 at 14:13 GMT
Yup, #1693 can use the implementation here. And then this issue could use the implementation for replace
#1697. Which is why I said they are related.
Comment by cmcqueen Friday Nov 12, 2021 at 06:54 GMT
I reckon ConfigArgParse package for Python is a terrific reference for a good useful and featureful implementation.
Issue by casey Sunday Nov 13, 2016 at 23:56 GMT Originally opened as https://github.com/clap-rs/clap/issues/748
I wrote up a sketch of how this might look here:
https://github.com/casey/clap-config
Basically, it would be pretty cool to generate a simple config file parser from a clap argument parser. There's a longer description of some use cases in the readme, but it would be great for projects which are configurable with both the command line and a config file, and for helping projects add config file configuration in addition to command line configuration.