Open tailhook opened 5 years ago
Thanks for bringing this topic up for discussion.
I agree that structopt
provides a nicer model to work with and I would be in favor of moving more towards that. But at the same time: I don't think that is a must. The degree of "pain" (for lack of a word with less negative connotations) of using argparse
directly depends on how many things you need it for. For small programs the existing model is fine, I would argue. And even for a program with a good amount of parameters and subcommands it's not the worst (some getopt
style solution is probably much worse). It's a compromise as so many things (which is also what I was getting at when saying that "it fills a niche") :-)
In nitrocli
, a project we currently use the crate in, we've gone through some effort to generate things such as enums used by argparse
and descriptions of parameters. Right now that is only happening through the usage of declarative macros and very limited. I plan to look into this a bit more and perhaps the results of that investigation could be of benefit here (although I should mention that this is not a priority of mine right now and I have limited time available for the project to begin with; I also don't necessarily want to go down the procedural macro route unless I see a big need).
That being said, I don't think points 1. and 2. are necessarily mutually exclusive. As you said 2. could easily be an addon, if, say, it was procedural macro based. As for point 3., it may be worthwhile getting in contact with the authors of clap
and structopt
to understand whether that is a concern of theirs, if they have done anything to do about that, and to see if the root causes are understood. Perhaps we should also first check if structopt
incurs the same binary bloat, which I assumed but did not verify (if the compiler is unable to eliminate much code for clap
for a simple project it appears unlikely that it will do so in the context of another crate built on top; but we should be sure).
As for clap
I am fairly confident that the author is interested in getting the size down as there are multiple articles on the subject (this one that I linked in the pull request already, for example). However, given what I have read so far I am cautiously pessimistic about the outcome. I shall inform myself some more on what has been done and where things are, though.
My suggestion would be as follows:
1) Check whether structopt
is as bloated as clap
(I would probably be able to report something on that front unless you want to look into that @tailhook)
2) Read up on clap
's optimization attempts and see if
2.1) The root cause of the bloat is fully understood
2.2) We can do anything about that
I plan on reading what the author of clap
has written up in full. Of course, it doesn't hurt if multiple people have that background.
3) At the same time (mainly for the reason that I don't think moving closer to the structopt
model is a must) I do think that https://github.com/tailhook/rust-argparse/pull/50 should be merged and a new version released, as it is a bug fix. Okay, that one is driven by my need for the fix :)
I have updated numbers now that include structopt
. Note that total numbers are smaller than those mentioned here https://github.com/tailhook/rust-argparse/pull/50 because I updated to Rust 1.32 which changes the allocator used (removing roughly 324 KiB from binaries according to measurements I've done in the past).
(unit is bytes)
With default settings: structopt: 862432 clap: 866528 getopts: 358624
When optimizing for size (opt-level = "z", lto = true, codegen-units = 1, incremental = false): structopt: 575712 (https://github.com/d-e-s-o/dictcc-cli/commit/6cad765c1f18267249eefeb09b158fcf1c4fb458) clap: 579808 (https://github.com/d-e-s-o/dictcc-cli/commit/0a48fe9579139970d1faea6a941a3da21e846007) getopts: 280800 (https://github.com/d-e-s-o/dictcc-cli/commit/06c7e738fa3637df426fd432d57529a4a2189f4d)
We can see that while structopt
does not add anything over clap
(in fact, it reduces the overall size slightly), it still plays in the same ball park. For small programs, I would argue, the increases that clap
and structopt
incur are quite significant.
It's still not clear how much argparse adds. In my expericence, it's ~100Kb, which isn't too small either :(
Other than that, I haven't had time to investigate it myself, yet...
It's still not clear how much argparse adds. In my expericence, it's ~100Kb, which isn't too small either :(
Yeah, I first wanted to address the structopt
situtation. Anyway, now tested with argparse
I get the following:
argparse: 301280 (https://github.com/d-e-s-o/dictcc-cli/commit/e64a968a661ba6bea71cf603f881b1880d26ae24)
So that's a bit more than getopts
but still significantly less than clap
or structopt
. I'd also not make this only about binary size. The number of dependencies (including transitive ones) is another factor influencing build time and maintenance effort (keeping up & auditing), among probably others. And while clap
/structopt
are not the worst offenders in this space (having less dependencies than other crates), they still have infinitely more than argparse
:P
I read through the two articles that the author of clap
has published. I doubt we can do much on this front (for the following I equate clap
and structopt
as the size of the former directly influences the size of the latter by virtue of structopt
just being a wrapper; we have seen very similar binary size characteristics).
The size truly seems to come down to code (.text
) and the author already has made effort to get this down. It seems unlikely that someone less familiar with the code base can change that much for the better, given that there was already work on this front that picked the low hanging fruits.
I like this crate more than structopt
, mainly because it's quite a bit more lightweight, and straight to the point, since it avoids dabbling in macros.
I am using this library primarily because clap
(and therefore structopt
) does not understand (and so cannot be used to implement) conventional Unix command line parsing. For example, it is usual for a Unix program to understand later options as overriding earlier ones, but clap treats them as conflicting. You can't even do something as simple as have a boolean --thing
--no-thing
pair where --no-thing
can be used to override an earlier --thing
. There are other anomalies and clap
in particular lacks the flexibility that argparse
provices.
I would love to have something more declarative, structopt-style. I don't care about the code size very much :-).
To go back to this library, my view is:
add_option
et al insists on taking &'parser str
for help text etc. This makes it more awkward than necessary to have programatically-generated strings. IMO these parameters should all be Into<String>
and argparse should make a copy. The extra heap allocations are trivial in cost compared to program execution so we should provide an easier interface and not try to hyperoptimise by taking references to avoid copies.Ref
) should be able to have different metavar
s.StoreOption
ought probably to be called StoreSome
.Some of these changes would be API breaks.
@ijackson I agree with most of what you have said. I think this was my first project in Rust and dates back to Rust ~0.7, so certainly needs updating. And I'm also not happy with some of the clap's limitations. However, I do not have enough time to work on this now.
@ijackson I agree with most of what you have said. I think this was my first project in Rust and dates back to Rust ~0.7, so certainly needs updating. And I'm also not happy with some of the clap's limitations,
Well I think you have done well for your first project then! Thank you :-).
However, I do not have enough time to work on this now.
I may find time to do so. If I do, should I submit MRs, or pick a new name and fork it, or what? I guess the first step would be an MR with a branch containing proposed changes.
You can't even do something as simple as have a boolean
--thing
--no-thing
pair where--no-thing
can be used to override an earlier--thing
.
BTW this can be done with overrides_with in clap.
I was used to thinking that this library is not needed anymore because clap and structopt do the job just well. But there are valid points by @d-e-s-o, the most important of it is that clap is very large in size.
On the other hand, I very much like the model structopt provides (i.e. parse in the structure instead of complex borrows). The actual semantics of structopt annotations are sometimes ugly though because it is a thin wrapper around clap (with different actual syntax).
So we have a few options: