Open GoogleCodeExporter opened 9 years ago
GoogleCodeExporter
commented
9 years ago Hm. I made the macro auto-stringify the argument for consistency with the flag
name, but I think I'm reconsidering the logic of that... having it take a const
char* directly might be better.Original comment by novas0x2a on 25 Aug 2010 at 9:24
GoogleCodeExporter
commented
9 years ago Thank you for the suggestion. I do agree this is something that wouldn't be
too hard to implement.
However, one of the goals with this flags package is to keep the API as narrow
as practicable. I'd rather not have two versions of every DEFINE macro.
I'm also not sure how you plan on using this category information. Is the idea
that you would just call GetCommandlineFlagInfo() on the flag you cared about,
and look at the category field manually, and do something with it? Or were you
planning to further modify the flags system in the future to do something with
the category?
If the former, it would be easy enough to just keep this information elsewhere,
outside the flags system (have a map from flagname to category that you
maintain). If the latter, I guess I'd like to see how you planned to use it,
but it would have to be pretty compelling to justify the API widening.Original comment by csilv...@gmail.com on 25 Aug 2010 at 10:52
GoogleCodeExporter
commented
9 years ago My ideal use case:
__file1.cc__
DEFINE_TAG_bool(Module1, foo, true, "Foo")
__file2.cc__
DEFINE_TAG_bool(Module2, bar, true, "Bar")
__app.cc__
static const char kTOOL = "app";
DEFINE_TAG_bool(app, "baz", false, "Baz");
DEFINE_TAG_bool(app, "qux", false, "Qux");
int main(int argc, char *argv[]) {
gflags::SetUsageMessage("[options]\n Frozzle a Gibble");
gflags::ParseCommandLineFlags(&argc, &argv, true);
return 0;
}
__OUTPUT for `app --help`__
app [options]
Frozzle a Gibble
Help Options:
--help
Show abbreviated help
--help-all
Show all options
--help-module1
Display help for Module1
--help-module2
Display help for Module2
Application Options:
--baz
Baz [default="false"]
--qux
Qux [default="false"]
__OUTPUT for `app --help-all`__
app [options]
Frozzle a Gibble
Help Options:
--help
Show abbreviated help
--help-all
Show all options
--help-module1
Display help for Module1
--help-module2
Display help for Module2
Module1 Options:
--foo
Foo [default="true"]
Module2 Options:
--bar
Bar [default="true"]
Application Options:
--baz
Baz [default="false"]
--qux
Qux [default="false"]
__END__
"Application Options" are the ones where flag.category() ==
ProgramInvocationShortName() || flag.isKey(). I have code for some/most of this
already, but I'm basically reimplementing all of gflags_reporting in my own
code to do so. It seems like this would be useful to others, but only if I
actually integrate it into gflags. Is this somewhere you'd like to go?Original comment by novas0x2a on 26 Aug 2010 at 7:16
GoogleCodeExporter
commented
9 years ago Good question. I'll ask the folks who have been working with key-flags here
(only in python for now, I think), and see what they think.Original comment by csilv...@gmail.com on 28 Aug 2010 at 5:19
GoogleCodeExporter
commented
9 years ago We've started discussions, and I'd like to understand a bit more how you'd use
this. What values would you use for the 'category' values? Would they really
just be replacements for the filename (so all flags in a given file would have
the same category name)?
What happens when you use a library that you didn't write, that happens to use
the same category name for a flag that you want to use in your app? Or if you
write a library that uses a category name, how do you make sure its name
doesn't conflict with other category names? This question makes more sense if
categories are meant to be descriptive ("debug" or "required" or "performance"
or whatever), rather than replacements for filenames.Original comment by csilv...@gmail.com on 31 Aug 2010 at 12:54
GoogleCodeExporter
commented
9 years ago In my particular case, I'll probably be using n categories for m files, where n
<< m, organized by source module (vwCore, vwFileIO, etc).
> What happens when you use a library that you didn't write, that happens to
use the same category name for a flag that you want to use in your app?
I just see that as an namespace/API problem that gflags doesn't need to try to
solve directly. If the library is using general names (read: "putting things
into the global namespace") then they need to put some thought into their
published interfaces, to allow downstream users to multiplex on the same flag
names (so parent and child library can both use FLAGS_debug, for example).Original comment by novas0x2a on 31 Aug 2010 at 1:31
GoogleCodeExporter
commented
9 years ago I see -- so for you, categories are a way to kind of 'group' files together, is
that right?Original comment by csilv...@gmail.com on 31 Aug 2010 at 1:44
GoogleCodeExporter
commented
9 years ago Yep!
One additional thought is that, perhaps one more type of crowd control might be
useful: marking a flag as boring (the opposite of key).
Flags start at 1.
1) +1 if flag isKey or flag.category() == ProgramInvocationShortName()
2) -1 if flag is boring
3) +1 for each flag in modulename if --help-modulename is passed
When listing flags (with --help):
if flag.interesting < 1:
skip printing flag. skip printing --help-category for empty categories.
elif flag.interesting == 1:
skip printing flag, but print --help-category entry
elif flag.interesting > 1:
print flag and category
That way the application (which should be the final arbiter anyway, being
closest to the end user) gets full control over the flags printed. Boringness
is naturally overridden by specifically requesting information on a flag
(though there's a hole in this state machine if flags in the same category have
different interestingness levels)Original comment by novas0x2a on 31 Aug 2010 at 2:13
GoogleCodeExporter
commented
9 years ago OK, I've got a design written up for how we might implement flag
categorization. But I've been swamped with other things and haven't had a
chance to start it yet. But it's on the plate!
If you're interested, here are the design requirements and ideas as I've
written them down so far.
---
Objective
The --help flag is not very useful for large binaries, which may
include hundreds of flags. --helpshort has less output, but
still mixes useful and less-useful flags, and can leave out
useful flags defined in libraries.
To improve the utility of --help, I propose adding flag
categories to gflags for c++. (Python already has a related
functionality with "key flags".) The intended use is that
file-writers can annotate how important their flags are by
assigning them to an appropriate category, and the --help system
can make use of this information to display the most important
flags in a more user-friendly way.
The category system is flexible enough to serve other uses as
well, though they are not an objective of the design. For
instance, even if two files are conceptually related, they will
show up in different sections of traditional --help output. With
categories, the flags in each file could be assigned to the same
category, and hence show up together in --help.
Design Highlights We will add new functionality to gflags.h to
assign a flag to a category. Categories are arbitrary
strings (though some category names will have special meanings to
the flags system). For v1.0, we will restrict each flag to being
in exactly one category. Flags that do not use this new
functionality to obtain a user-defined category, will be assigned
their filename as their category. In this case, we say the flag
is in its "default category." Registering Categories We will
choose between one of these four possible mechanisms for adding a
category.
1. Add a function: RegisterCategory(&flag, "category_name").
This is analogous to to the current RegisterFlagValidator().
As with RegisterFlagValidator, RegisterCategory will require a
dummy variable so we can call this function at global
initialization time.
2. Add a macro REGISTER_CATEGORY(flag, "category_name"). This
is similar to above, but has a prettier syntax (it creates the
dummy variable for you, etc.).
3. Add a new series of macros: DEFINE_category_bool(flagname,
default value, category, description),
DEFINE_category_int(...), etc. Have folks move to these new
macros for new code.
4. Use some preprocessor tricks to allow for "variadic
macros," where folks can add a category name as an optional
third argument to DEFINE_bool, etc. So DEFINE_bool(flagname,
dflt, helpstring) will continue to work, but
DEFINE_bool(flagname, dflt, category, helpstring) will also
work, and add the given flag to the given category.For the
curious, here's how the trick would work:
#define CATEGORY_NAME(a, b, c, ...) c
#define HELP_STRING(a, b, c, ...) b
#define DEFINE_bool(flagname, dflt, ...) \
const char* category = CATEGORY_NAME(__VA_ARGS__, __VA_ARGS__, NULL); \
const char* helpstring = HELP_STRING(__VA_ARGS__, __VA_ARGS__); \
etc.
I think (1) and (4) are the best options. I like (4) because one
day, in an ideal world, we'll move every flag over to the new
scheme, and at that time the macros will have the name we
want (unlike with (3)), without having to have a big flag
day (pun intended) to rename all these macros. (1) fits well as
it parallels existing functionality with the flag validators.
Arguing against (1), the flag validators aren't used very often, and I think
one reason why is the awkward syntax for using them. So I'm leaning towards
(4).
Changes to --help
--help will change to group flags by category instead of by
filename, as it does today. For flags not in their default
category (of the filename they're defined in), we will augment
the --help output to say what file the flag is found in.
The order we display the categories will be hard-coded. Some
names -- to be determined -- will be hard-coded by the flags
library to display first: "required", perhaps "important", etc.
Then all remaining categories mentioned in the same file as
main() (including the default category of the 'main'-file
filename) will be listed, in alphabetical order. Then all
remaining user-defined categories will be listed, in alphabetical
order. Finally, all default categories will be listed, in
alphabetical order.
All existing --help variants that work on
filenames (--helpmodules, for instance) would change semantics to
work on categories instead. We may add --helpcategories as an
alias for --helpmodules.
A property of this design is that until explicit categorization
is done, the output of --help will be exactly the same after the
change as before.
Based on user feedback, we may change --helpshort from its
current behavior, to new behavior of only listing flags in
certain categories: "required", "important", etc. Once all
important libraries and binaries have had their major flags
categorized, we will consider changing --help to act the same as
--helpshort. --helpall will be added as a way to get the old
behavior where all flags are listed.
The Category Taxonomy
We will have to decide on some common categories that all apps
can use. We will want a mix of category names that range from
critical flags (app won't run if these aren't set to a
non-default value) to totally unimportant flags (deprecated flags
that don't do anything), with several levels in between. We may
also want some flags based on usage: for instance, "network
performance tweaking".
Applications can also develop their own categories, but since
category names are global, they will probably want to include the
application identifier, or some other unique token, in the
category names.
We will restrict category names to being printable 7-bit ascii.
We may restrict further to [-_A-Za-z0-9 ].
This section should expand with the list of category names, as we
develop them.
Overriding Categories
(This idea is taken from python key flags.) It should be
possible for an application to override the category decisions
that a library makes. For instance, library X may decide its
flag --compress_frobs is a performance flag (turn it off for
faster performance). However, if a particular app never uses the
frob feature of library X, or only uses it once for a small
amount of data, --compress_frobs would be a useless flag for that
app. So it should be possible for an app -- a file containing
main() -- to override library determinations of categories.
For v1.0, we'll implement a simple scheme where an app can change
the category for all flags in a given file or directory. We'll
add a new function ChangeFlagCategoriesTo("category",
path_substring). (Could use a better name.) This will cause the
flags system to iterate through all flags its seen. For every
flag that's found in a file that has "path_substring" as a
substring, it changes the flag's category
to "category". "path_substring" will typically be a file or
directory name.Original comment by csilv...@gmail.com on 14 Oct 2010 at 1:26
GoogleCodeExporter
commented
9 years ago Hi,
I'd really love to see this implemented and released as it would be currently
the major reason for us not to adopt gflags. (We need to have a --help output
which users who are less into software development can grasp without getting
distracted by module names and less important flags.) From the design of how to
associate category names with flags, I would also say that (4) is the best
solution. However, I would also suggest to implement (1) as an option to
override the category assignment.
For example, consider a flag that was defined in a library such as glog. Here
the developer of glog will decide for a category name. If I am as user of this
library am not happy with this assignment or want to put those flags in a group
of my own, e.g., together with some of the flags defined in my own modules,
where I want to use a different category name because the one chosen by the
glog developer is not meaningful enough for my flags, I would need to assign a
new category name to those flags.
As (1) and (4) both would do the assignment during static initialization, I
wouldn't be too certain about the order of code execution. But as we talk about
categories which are mainly if not only used for the reporting system, it might
be reasonable to have the RegisterCategory() function not be called at static
initialization, but simply inside the main() function before the parsing of the
command-line.
Example usage:
io.h:
DECLARE_string (format);
io.cc:
DEFINE_string (format, "nifti", "File formats", "Select format for output image
files.");
main.cc:
#include "io.h" // defines flag "format" in category "File formats"
// category names
const char *kIoFlagsCategory = "Input/Output options";
// define some more flags in the main module
DEFINE_string (in, "", kIoFlagsCategory, "Input file path");
int main (int argc, char *argv [])
{
// change category of flags defined in module foo
RegisterCategory (format, kIoFlagsCategory);
// parse command-line
ParseCommandLineFlags (&argc, &argv, true);
// ...
return 0;
}
Original comment by andreas....@gmail.com on 11 Aug 2011 at 4:38
GoogleCodeExporter
commented
9 years ago Wow, this bug is coming up on its year anniversary!
We finally got someone within google interested in implementing this, or at
least a portion of this, so it may be coming. Of course, there are lots of
issues still to design, so it may be slow going.
I think the plan is to go with (4), like you liked too. Keeping (1) as an
option to override (4) is interesting. I think you're right it would make more
sense as a function call you do in main before flags-parsing (just like you can
override the default flag value in main before flags-parsing). We probably
wouldn't do that in v1, just to keep the API as small as we can, but would look
to see if there's a need to add it afterwards.
Original comment by csilv...@gmail.com on 11 Aug 2011 at 9:01
GoogleCodeExporter
commented
9 years ago I started the implementation, and immediately got some pushback from some
relevant folks (big users inside google), who are not convinced about the
complexity/benefit trade-off. So the "slow going" is an accurate prediction.
:-( I'm keeping this bug open, so we can hopefully figure a good solution for,
at least, the original request, if not for flag categories in general.Original comment by csilv...@gmail.com on 25 Aug 2011 at 11:56
GoogleCodeExporter
commented
9 years ago Hi, I just wanted to note that just after my first comment here, I extended the
shFlags written by Kate to account for categories as well as to clean up the
--help output and adding --helpshort and --helpxml similar to the C++ and
Python solutions (some modifications might be a matter of taste, though)...
sure this is another project and I probably should get in touch with Kate to
see what she'd like to incorporate into the shFlags project.
Next I was about to touch also google-gflags as well as python-gflags to make
all three options parsing solutions more similar regarding use as well as look
and feel (including support for custom categories to give the user a chance to
arrange the output; yet, I think the three independent projects are kind of
"out-of-sync"). I would be open for any teamwork as well as contributions to
the "official" open source Google projects.Original comment by andreas....@gmail.com on 28 Aug 2011 at 4:35
GoogleCodeExporter
commented
9 years ago You're definitely right that c++ flags and python flags have diverged a little
bit. In some sense I think that's appropriate, since the languages are
different (for instance, we aim to provide flags for each 'built-in' type,
approximately, but the set of 'built-in' types is much larger in python than in
C++). That said, there are some differences that aren't motivated by language,
like the python-only 'key flags'.
I'm not sure how much support there would be for changing the API for any of
these at this point though, even to gain consistency. I guess it would depend
a lot on the specific details.Original comment by csilv...@gmail.com on 29 Aug 2011 at 11:08
GoogleCodeExporter
commented
9 years ago I understand your point. Certainly, the priorities of Google and us (a research
group at the University of Pennsylvania) differ. Thus, I will be working on our
own fork of these libraries for our internal use and once it's stable we will
see how we can best make it part of the public domain.Original comment by andreas....@gmail.com on 30 Aug 2011 at 3:46
GoogleCodeExporter
commented
9 years ago Custom flag categorizes which are used in-place of per-module help output are a
feature which I believe is one of the most important for users of the library.
Comparing it to other command-line flag parsing libraries, only gflags has this
quite developer-oriented help output with C++ module file paths and such,
whereas the help output of a program is generally meant for users of a
software, not those that develop it.
A solution to this issue should also consider the comments and suggestions of
issue 32.Original comment by andreas....@gmail.com on 20 Mar 2014 at 3:56
Original issue reported on code.google.com by
m...@fluffypenguin.orgon 25 Aug 2010 at 1:48Attachments: