Auto-generate manpage, help docs, etc.

joshtriplett commented 8 years ago

Maintainer notes:

Blocked on https://github.com/clap-rs/clap/issues/2914 for decoupling help information gathering from formatting
help2man can be a source of inspiration for how to integrate this into a users process

I'd love to have support to generate a manpage. This would use a mechanism and infrastructure similar to #376. Additional functions to override or augment portions of the generated manpage could come later, but I think with relatively few additions this could become an incredibly useful mechanism.

The manpage title should default to the bin_name value.
The section should default to 1.
The NAME section should default to bin_name \- about, where about is the string set by .about.
The SYNOPSIS section should contain the usage strings for the command and every subcommand.
The DESCRIPTION section would need some new paragraph-style information provided (also usable as a more structured .before_help).
The "OPTIONS" section should document the flags and args for the top-level command.
If the command has subcommands, a "SUBCOMMANDS" section should document each subcommand in a sub-section.
The AUTHORS section should contain the author information, if provided.
The SEE ALSO section would need some new mechanism to populate it.

I'd be happy to help with manpage markup, once I see the details of the mechanism used in #376.

kbknapp commented 8 years ago

I like this idea! I'll have a better idea about what all it would take once I finish #376 but I think it could be done. And if nothing else, it'll at least give a really good starting point that one could then tweak manually with little effort.

joshtriplett commented 8 years ago

Absolutely. I'm also hoping that, similar to help2man (which doesn't handle most of what clap can do), this could support providing arbitrary additional documentation to integrate into the generated manpage. That would allow maintaining information in only one place (such as options and their documentation).

hgrecco commented 8 years ago

I was thinking about the same thing recently and I think we can reuse much of the infrastructure done for the templated help. But I would also like to propose a different way to generate the man page (and access the completion). I am opening another issue for this.

kbknapp commented 7 years ago

Addressing this issue soon. I'd like to get the ability to generate man pages, help docs, etc. For the help docs, I'd especially like to be able to generate based off a template exactly like App::template works. Except recursively going through subcommands. The template would allow doing things like markdown, etc.

Also, for help docs I'd like to choice to split the files or use a single document.

matthiasbeyer commented 7 years ago

As far as I can see, this was moved to the 3.x release for clap? Either way, I'd like to pronounce interest in this feature. Not only for git-dit but also for imag.

kbknapp commented 7 years ago

@matthiasbeyer yes, this is a feature I want, but need to get 3.x out the door first because otherwise it'll just keep getting pushed off and pushed off.

valpackett commented 6 years ago

Since clap has plenty of information about the structure of commands and args and whatnot, it should be possible to build pages in the mdoc language, which is semantic (i.e. it has entities like "flags" and "commands" instead of just "bold text" etc.)

yoshuawuyts commented 6 years ago

Heya, on the CLI WG repo we've been working on this! It's a bit rough, but I reckon we might be able to create a compelling story!

https://github.com/killercup/roff-rs/ exists to generate .troff files. It's a bit versose, so we built
https://github.com/rust-clique/man The goal is to have this generate (structured) man pages. We're mostly just waiting to merge https://github.com/rust-clique/man/pull/6 before it should be usable!
We also have an initial implementation of clap v3 -> man page. Perhaps it would make sense to move it into a separate repo first, and if it works out alright we could eventually stabilize it and move it into Clap.

Screenshot

Structured man page, generated by the `man` crate

2018-07-19-141825_1920x1080

Hope this is all useful. Excited to have man page support for Clap!

kbknapp commented 6 years ago

@yoshuawuyts this is awesome! My thoughts were to place the manpage generation into the clap_generate crate (I'm not opposed to a rename if there is something more fitting) along with the shell completion script generation since they're doing very similar things.

If you'd like I'd be more than willing to add people to the org and repo! So we can make it official.

yoshuawuyts commented 6 years ago

@kbknapp oh awesome, that def seems like the right way forward! Would be happy to join in!

Also cc/ @spacekookie here (she wrote all the clap v3 -> man glue code). Perhaps you would be interested in joining?

killercup commented 6 years ago

@kbknapp not sure if you've seen it yet but I spend an hour yesterday to throw https://github.com/rust-clique/clap-md together -- its goal is to render Markdown documentation for clap applications. Feel free to move this to clap-generate, too! (I'd be sad to see that crate name go, though)

kbknapp commented 6 years ago

@killercup I love it! This is something I've been thinking about in the back of my mind that I'd love to put some time into! I've been passively looking at things like flatdoc and how other projects with large CLIs have provided docs (ones like Docker, etc.) to see if there is something we could use. But it's been on the back burner with this 3.x work :stuck_out_tongue_winking_eye:

I've sent out the invites to the clap-rs org to the three of you above, and yeah I'd love to move that to the org as well!

vn971 commented 5 years ago

Folks, how can I generate man pages from clap now? Is it possible?

ssokolow commented 5 years ago

I actually just accomplished a non-pure-rust version of that last night in the rewrites I did for my rust-cli-boilerplate.

Basically, I tweaked the output from StructOpt to properly match Linux platform conventions (eg. setting author="" and starting about with a newline so the <name> <version> line doesn't get wrapped into the beginning of the description text) and then ran help2man -N on it.

If you want to poke at it, it's just dist-supplemental or the just dist and just install commands which depend on it. (They'll also build and, if requested, install completions for zsh, bash, and fish.)

(My next goal will probably be to clear out that "build and publish a bundle of Clap validators for common cases" TODO at the bottom of the readme.)

ssokolow commented 5 years ago

Since I didn't think a boilerplate template necessitated screenshots in an already quite long README, here's what it looks like when I run ./apply.sh ../boilerplate; cd ../boilerplate; just install; man boilerplate: screenshot2

EDIT: And I just realized that I forgot to make dist-supplemental guarantee that the binary had already been built before calling help2man. Fixed.

pickfire commented 5 years ago

@ssokolow I think those generated man pages are using too much spaces, can you please try it with 80 columns for your terminal? It may look ugly on some terminal with lesser width though.

vn971 commented 5 years ago

@ssokolow will it work correctly with subcommands? The way --help works on clap now, it will just list the subcommands, but it will not show subcommand-specific keys and arguments. This will lead to incomplete man if piped to an external process directly. Are the subcommands currently addressed in your project?

ssokolow commented 5 years ago

@pickfire Here's a screenshot at 80 columns (as lazily determined by running print 'x'*80 in a Python REPL and then resizing the window to just fit it without wrapping)

screenshot2

@vn971 I hadn't gotten around to trying to accomodate subcommands yet because I'm using my own needs to set the priority of various features and I don't use subcommands often.

For example:

At the moment, I'm about to rewrite apply.sh in Python, add a config file, and have the default config file run xdg-terminal, xdg-open src/main.rs, and git gui after creating the new project since that's the current number-one thing I could do to reduce my urge to turn to Python rather than Rust for quick little throwaway scripts which will turn out to be anything but. (I'll bundle copies of the XDG utilities as fallbacks that don't get copied into new projects.)

(I'm used to running gvim script_name.py in my Quake-style terminal followed by boiler\c, then bringing git into the mix when I realize it's not going to be a throwaway. This would solve that while making it even more convenient.)
I'm planning to experiment with a quick custom derive that allows me to hang StructOpt and something like config-rs off the same struct, then call serde_output.merge(structopt_output) as an "I'm tired of waiting for a proper solution and I think ripgrep's solution is ugly" way to get config files without duplicating the schema.
Both as a convenient reference for myself and as part of my commitment to good documentation, I'm planning to rename the validators after their intended uses (ie. output_file_path rather than path_valid_portable) and include a block like this at the top of each validator's rustdoc page:
Conventions::
- Use -o to specify the output path [1]
- Interpret a value of - to mean "Write output to stdout" [2]
- Because -o does not inherently indicate whether it expects a file or a directory, consider also providing a GNU-style long version with a name like --outfile to allow scripts which depend on your tool to be more self-documenting.

pickfire commented 5 years ago

@ssokolow Nice. Maybe you can just use env MANWIDTH=80 man boilerplate.

ssokolow commented 5 years ago

That said, I just noticed that, when I call help2man now, I'm getting a doubled body on the manpage, so I'll have to git bisect to find the cause tomorrow.

I'll also add a test for that odd behaviour to my test suite for the build automation.

ssokolow commented 5 years ago

Found the problem. I had a "Durr" moment when writing the justfile.

Originally, I'd accidentally written cargo run -- help, which worked because help2man adds --help and --version, which causes the help to have no effect.

Later, I "fixed" it to cargo run -- --help, so help2man was running boilerplate --help --help and boilerplate --help --version to extract the info.

I've pushed a fix and added a regression test to the test_justfile.py I use to make sure I don't screw up my justfile while refactoring.

(On a related note, I spent yesterday's coding time putting together a gen_justfile_reference.py script which is now responsible for generating and injecting the reference tables in the README.md by parsing justfile. I'll probably rewrite these helpers in Rust once I've moved the project template into a subfolder.)

pickfire commented 5 years ago

@ssokolow Just wondering, how does boilerplate generate the ENVIRONMENT section in man page?

ssokolow commented 5 years ago

At present, rust-cli-boilerplate simply doesn't (generate an ENVIRONMENT section) and I don't see how it could automatically extract that information, given how the it doesn't currently do anything with the environment.

(ie. In its current stage of development, loading configuration beyond command-line arguments is up to you, so documenting them is also.)

However, it's trivial to add an --include or --opt-include call to the help2man line in the justfile to append a chunk of raw *roff text for an ENVIRONMENT section and I'll consider adding an --opt-include (include if exists) by default to broaden the scope of people who can use the justfile without modification.

In the longer term, once I've got unified configuration of some sort, I'll look into doing something similar to --dump-completions to allow the program to report the environment variables it obeys so I can have a build script generate said *roff text.

pickfire commented 5 years ago

However, it's trivial to add an --include or --opt-include call to the help2man line in the justfile to append a chunk of raw *roff text for an ENVIRONMENT section and I'll consider adding an --opt-include (include if exists) by default to broaden the scope of people who can use the justfile without modification.

Would not that be limited to just *roff? How about mandoc and friends?

ssokolow commented 5 years ago

The man command is a *roff renderer. --include and --opt-include literally just copy the contents of the files you specify into the final output.

mandoc and friends, by definition, must output *roff markup for man to understand it, so you can use --include or --opt-include with their output.

(It's like something which generates HTML docs providing an option to copy blocks of raw HTML into the final output.)

dashohoxha commented 5 years ago

I am not sure whether this is useful, but in the past I have used http://rtomayko.github.io/ronn/ for writing man pages. They look like this:

ssokolow commented 5 years ago

I'm not really a huge fan of adding more Ruby dependencies. I'd probably do something with rust-cli/man instead.

(But thanks for the effort nonetheless.)

pickfire commented 5 years ago

Would it be good to generate the commands output to rustdoc as well? The CLI docs can be viewed in rust offline docs as well.

It could take advantage of the existing comments (which rustdoc already does) in addition to the extra commands, seems useful in context like https://github.com/mozilla/neqo/blob/6c012fb3416586b7c9604348b128e4cffc250338/neqo-server/src/main.rs#L26-L28

ssokolow commented 5 years ago

I'd need to see a mockup of the kind of thing you'd expect to generate, but my instinct is "Yes, it would be good".

Dylan-DPC-zz commented 5 years ago

Have you looked at https://github.com/rust-cli/man?

ssokolow commented 5 years ago

@Dylan-DPC

Already been mentioned.

ssokolow commented on Mar 28 I'm not really a huge fan of adding more Ruby dependencies. I'd probably do something with rust-cli/man instead.

(But thanks for the effort nonetheless.)

pickfire commented 5 years ago

@Dylan-DPC I did saw that but if I am correct you are replying to https://github.com/clap-rs/clap/issues/552#issuecomment-534080103, the context was generating help similar to man pages but to rustdoc instead since rustdoc is not fully utilized for binary packages as of now. Maybe I should go ask in #rustdoc.

msarfati commented 4 years ago

After working a bit on this problem on my own, I wonder if the best solution for man-page autogen is to support the ability to convert a clap::App to YAML, after it's been built using its builder pattern, and then just parse the YAML into man / *roff with rust-man. This feature can also help users migrate from the builder pattern to YAML.

I would be very interested in working on this and maintain it part of Clap (and not my own unicorn fork of Clap), but I'm not really sure how the Clap team feels about introducing a feature like this, or how to go about pitching this. Also would like to know if anyone else is working manpage autogen?

ssokolow commented 4 years ago

Waitaminute.

When did YAML come into the picture? I'm very much against using YAML in new code for a variety of reasons that boil down to how the spec's hugely overcomplicated from trying to be too many things for too many different people (longer than the XML spec and doesn't even achieve all those goals), it's got various lurking footguns such as confusing an unquoted string literal for Norway's country code (no) with an alternative "do what I mean" representation of boolean false, and different YAML implementations seem unable to agree on how to implement it.

This has caused real-world problems in the past. (Not even counting the various problems caused by libraries implementing insecure by default decoding APIs.)

It's also at odds with the Rust philosophy when it comes to footgun-encouraging design, reminding me more of CoffeeScript's use of whitespace and PHP's weak typing.

Any chance you could bypass the serialization/deserialization step entirely or at least use JSON or TOML instead?

CreepySkeleton commented 4 years ago

ability to convert a clap::App to YAML, after it's been built using its builder pattern, and then just parse the YAML into man / *roff with rust-man.

If you mean https://github.com/rust-cli/man by rust-man here than I think the most sensible approach here is to convert clap::App into man::Manual directly, without any intermediate formats, YAML or not YAML.

Serializing App to / deserializing Manual from some config file can be implemented separately.

The only thing is that https://github.com/rust-cli/man is kind of dead as I see it (a year of no updates). cc @rust-cli

Any chance you could bypass the serialization/deserialization step entirely or at least use JSON or TOML instead?

I'm seconding @ssokolow here, YAML is not a good choice for config files.

msarfati commented 4 years ago

Those are fair criticisms of my YAML idea @ssokolow and @CreepySkeleton -- I only proposed YAML because it was something already supported by clap (albeit in a different use-case), but that approach indeed has its issues.

I suppose I could bypass serialization/deserialization entirely and just use a clap::App to construct a man::Manual in rust-cli/man. It seems like this approach might involve exposing some of clap::App's attributes as public, or writing new getter functions/macros for accessing some of clap::App's attributes not currently exposed. I'll get to work on a prototype and get a better sense of what's required.

CreepySkeleton commented 4 years ago

. It seems like this approach might involve exposing some of clap::App's attributes as public, or writing new getter functions/macros for accessing some of clap::App's attributes not currently exposed.

Those fields are pub but #[doc(hidden)] in both App and Arg. While I don't think that relying on such fields is fine for a release of minimal viable product, I think that this is acceptable for a prototype. So go ahead and "draw a sketch"!

spacekookie commented 4 years ago

The only thing is that https://github.com/rust-cli/man is kind of dead as I see it (a year of no updates). @rust-cli cc

Hey!

Indeed we haven't been working on this stuff actively in the last year, but we're getting the CLI WG started again. There's regular meetings (every first Monday of the month), and notes on the meetings here: https://hackmd.io/NKG9jUiFThGqgAqhsIniyg

Truth be told, we've been thinking about how to generate documentation in general, and have been looking at yamr and toml as formats. Toml is "simple" in the sense that it quite awful to write and structure. We wanted to put some tech demos together and announce them soon.

I encourage you to join the next meeting if you have feelings about this. We also have a ML now that you can add yourselves to via the rust/teams repo.

pksunkara commented 4 years ago

I agree with @CreepySkeleton here that we can easily create a to_man fn in clap::App that converts it into man::Manual. Clap already takes in config file, we don't need to create one more.

@spacekookie Thanks for the info on the meetings. Looking forward to join the next one.

We recently got a stream created in Zulip chat too #wg-cli

spacekookie commented 4 years ago

I agree with @CreepySkeleton here that we can easily create a to_man fn in clap::App that converts it into man::Manual. Clap already takes in config file, we don't need to create one more.

I have to strongly disagree here.

There's a few problems with just having the documentation in code. For one it's impossible to translate, which is a huge problem at the moment, on the other hand a man page can't just be generated from a clap::App definition. A manual page is more than just listing the options from --help in a slightly different format. In fact, those pages exist sometimes, and they're usually useless. Instead, a manual page needs to go into a lot more detail about the semantics of a tool. This is something that requires free form text and is best done in external files.

The exact format of these files, I'm not sure about. We could even have a folder with documentation, which is only supplimented with a config to put it all together, but it can't just be derived from the clap::App definition.

We recently got a stream created in Zulip chat too. https://rust-lang.zulipchat.com/#narrow/stream/220302-wg-cli

We decided on chat being on matrix in a past meeting (matrix.org:#rust-cli). I'm not sure there's enough volume to fill two chat channels, but in any case, do be aware that there's also discussions going on in the matrix channel.

vn971 commented 4 years ago

To correct a bit, the room id is #rust-cli:matrix.org.

CreepySkeleton commented 4 years ago

@spacekookie Totally agree on the "translations" thing, I didn't think about it, oops.

(I must confess that I'm no expert in man pages and if you think something of the following is wrong - tell me and the apologizing cat will apologize).

A manual page is more than just listing the options from --help in a slightly different format

The funny thing is that the man pages are exactly "the options from --help in a slightly different format" with some sort of header and a footer.

Let's look at the gcc man page - header + options + footer. git man page - header + options + footer. (This case is pretty interesting since they use two subcommand sections instead of just one, but still.) curl man page - header + options + footer. ...

I have yet to see a man page for a binary that stands out too drastically.

My core point here is that at least the "option" part can very well be generated from --help.

a folder with documentation, which is only supplimented with a config to put it all together,

For project-level man pages that cover multiple binaries - maybe. For a single binary? Not sure, but leaning to "one config file".

spacekookie commented 4 years ago

@spacekookie Totally agree on the "translations" thing, I didn't think about it, oops.

I think this is quite an important point that a lot of people forget about, which is a problem. And whatever tooling we have needs to address this.

A manual page is more than just listing the options from --help in a slightly different format

The funny thing is that the man pages are exactly "the options from --help in a slightly different format" with some sort of header and a footer.

To rephrase my initial point: content can't just be derived from the clap definitions, structures possibly could but I don't really see the point. Yes, man pages are usually "just" the set of options, but with significantly more explanation and cross-references to other man pages (see the git manuals for referenc).

I think it's important to keep the actual text outside of the Rust code for a few reasons. One is that we want to encourage people to write longer sections to explain their program, and doing so in a format that is optimised to convey emphasis, such as markdown, restructured text, etc is much better that having to write it in multiline strings in Rust files. Two is that we want these files to be translatable.

So considering that these files will need to follow some kind of structure anyway, to not create a huge mess between different language versions of the same texts, I don't really see the need to generate anything from clap. Rather, we would associate a documentation key with the option names or help text in clap. The idea is that this tool could be used in far more applications than just clap CLIs, and also allow other argument parsers to implement the same trait to benifit from translations, man page support, etc.

I think this is a far more elegant solution to this problem than having to couple it to one particular argument parser library.

Are you subscribed to the mailing list yet? I recommend you do that (over at the teams repo) and also check out the meeting notes we took in the last 2 meetings: https://hackmd.io/NKG9jUiFThGqgAqhsIniyg

pksunkara commented 4 years ago

I think you misunderstood my intention. You are right that we don't want to couple the man pages generator to clap. What I meant is we can use doc comments in the CLI built using clap_derive to generate default man pages which the author can override later if he wants.

I think it's important to keep the actual text outside of the Rust code for a few reasons. One is that we want to encourage people to write longer sections to explain their program, and doing so in a format that is optimised to convey emphasis, such as markdown, restructured text, etc is much better that having to write it in multiline strings in Rust files.

Well, doc comments is already a standard here.

spacekookie commented 4 years ago

Well, doc comments is already a standard here.

I really don't think this is a good way to go about this. Doc comments are convenient for documenting actualy functions, but now writing a book, for example.

Anyway, the clap_derive can do whatever, really. The idea is to have a "Doc" Trait, that allows tools to hook into sources and sinks, which means that you can have your doc comments if you want them, and then swap out the comments later for actual docs.

I personally don't think this is a good way about writing an application, because it depends on editors having good support for editing doc comments (not all of them do), and makes translating impossible, but obviously that's just my feelings about how software should be written ;)

ssokolow commented 4 years ago

Bear in mind that, on the Python side, I'm just finishing up a migration to Sphinx and the number-one factor that came into play on multiple occasions is internal consistency.

What is the number-one thing I hate about Sphinx's API documentation support? That the best I can do for ensuring nothing gets missed in refactoring is to babysit one foo.rst file per source file containing an .. automodule:: foo directive.

Doing so constrains my ability to document things compared to using more fine-grained autowhatever directives, but my number-one concern is minimizing the ability for things to drift out of sync because, in practice, I've run into too many Python libraries where I had to read the source because the Sphinx docs were incomplete somehow.

(I'm upgrading from an unmaintained Python 2.x documentation tool that behaves more like rustdoc.)
How did I handle the command's manpage? I chose a third-party .. autoprogram directive because, as constraining as it is to just use a stupid "Generate it from the argparse.ArgumentParser instance", I value guaranteed consistency between my parser and my manpage far more highly than being able to put in more details. (Note that argparse.ArgumentParser doesn't expect any kind of rich markup as input.)

If you force me to do too much babysitting of my documentation to, I'll just stick to my current solution of tweaking the clap template to bring it into compliance with GNU conventions and then running help2man after compilation.

(At minimum, have some kind of lint which complains if the data from clap appears to not line up with the externally provided data.)

Also, note that, while it requires raw *roff input, help2man provides a potential avenue for allowing external content to integrate with generated content.

INCLUDE FILES
       Additional material may be included in the generated output with the --include and --opt-include options.  The format is simple:

           [section]
           text

           /pattern/
           text

       Blocks of verbatim *roff text are inserted into the output either at the start of the given [section] (case insensitive), or after a paragraph matching /pattern/.

       Patterns use the Perl regular expression syntax and may be followed by the i, s or m modifiers (see perlre(1)).

       Lines before the first section or pattern which begin with `-' are processed as options.  Anything else is silently ignored and may be used for comments, RCS keywords and the like.

       The section output order (for those included) is:

           NAME
           SYNOPSIS
           DESCRIPTION
           OPTIONS
           ENVIRONMENT
           FILES
           EXAMPLES
           other
           AUTHOR
           REPORTING BUGS
           COPYRIGHT
           SEE ALSO

       Any [NAME] or [SYNOPSIS] sections appearing in the include file will replace what would have automatically been produced (although you can still override the former with --name if required).

       Other sections are prepended to the automatically produced output for the standard sections given above, or included at other (above) in the order they were encountered in the include file.

       Placement  of  the  text within the section may be explicitly requested by using the syntax [<section], [=section] or [>section] to place the additional text before, in place of, or after the default
       output respectively.

If I had a solution which auto-generated from Clap but also looked for .md override files with those [<section], [=section], [>section] or /pattern/ references, and it complained if they didn't match or matched more than once without some kind of "This one is expected to match regex repetition specifier times", I'd be happy.

codesections commented 4 years ago

@ssokolow I have a genuine question that I hope doesn't come off as snarky—I really am curious. You wrote:

If you force me to do too much babysitting of my documentation to, I'll just stick to my current solution of tweaking the clap template to bring it into compliance with GNU conventions and then running help2man after compilation.

Why do you want to have a man page at all, if you'd be happy with it duplicating the --help output? As an end user, I nearly always consult --help first and then, if my question wasn't answered, check the man page. Given that, I'd strongly prefer no man page to a man page that duplicates the --help output. In either case, my question isn't getting answered, but at least No manual entry for <COMMAND> doesn't cause me to waste time figuring out that the man page has no new info.

Do you have a use case for man pages where a man page that copies --help output is useful rather than annoying?

@CreepySkeleton wrote:

The funny thing is that the man pages are exactly "the options from --help in a slightly different format" with some sort of header and a footer. Let's look at the gcc man page - header + options + footer.

That isn't how I'd describe the gcc man page at all. (At least on my system. The man page I have is very similar to the online one. Is that what you were talking about?)

I'd describe the gcc man page as a 15,746-line document that extensively documents the gcc command. This includes several sections before documenting options, and an OPTIONS section divided into several subsections, and then several sections after OPTIONS. Within its OPTIONS section (which contains multiple subsections not documented via --help), it documents many options that are not documented in the (61-line) --help output. Even when options are documented in both the man page and the --help output, their description in the manual is much more detailed than their description in the --help output.

My core point here is that at least the "option" part can very well be generated from --help.

Do you mean that the OPTION section of the gcc man page could be generated from the --help output? If so, I disagree – the OPTION section is far, far longer than the --help output and includes options not documented in via --help.

Or do you mean that a list of options could be generated from --help, and that list could form a framework for an expanded OPTIONS section in the man page? If so, I agree. The man page will eventually be much longer than the --help output, but starting with a list of all options/flags documented in with --help seems like a helpful starting point/way to avoid inadvertently omitting some options. On the other hand, I don't think that starting with the --help descriptions for options is all that useful – in my view, the man page descriptions should be significantly more detailed.

CreepySkeleton commented 4 years ago

Fair enough, I think I was wrong. I would still like to see the relationship between options is documented in some standardized way(conflicts, one option implying another...), along with default values, possible values and so on. I agree that the detailed description should not be the same as in --help but I'm very convinced that the stuff above must be derived from the actual implementation to ensure they never get out of sync.

ssokolow commented 4 years ago

@codesections

There are several important details:

There's a surprising amount of inconsistency about how programs implement built-in help. I've actually got a Zsh script, the whole purpose of which is to veto GNU's decision that -h should not be an alias for --help. Heck, I can't remember which, but I think I even ran into some programs where they didn't implement -h or --help (They might have been using -? or X11-style -help) and did something unwanted and annoying to clean up after if you tried to ask for help that way.
As primitive as *roff markup rendered in man is, --help output from pretty much anything other than a program specifically configured to use clap's coloured output option is worse. (Plus, there's a nifty hack which overrides the terminal escapes used in man to get colourization reminiscent of what clap's colourization does.)
man command is more concise and easier to remember than command --help | less, especially if I then decide that, oops, I need less so I can use / searching.
While I'm not using it yet (partly because I hate *roff markup), help2man allows you to add blocks of information above, below, or within your autogenerated stuff.

My mention of rustdoc vs. Sphinx was carefully chosen. I want a solution where omitting an entry can only be done intentionally and it's impossible for the auto-generated parts to fall out of sync with reality unless you are specifically overriding them with manual bits rather than just augmenting them.

codesections commented 4 years ago

@CreepySkeleton

I would still like to see the relationship between options is documented in some standardized way(conflicts, one option implying another...), along with default values, possible values and so on. I agree that the detailed description should not be the same as in --help but I'm very convinced that the stuff above must be derived from the actual implementation to ensure they never get out of sync.

Agreed 100%

@ssokolow, thanks for the detailed reply. As mentioned upthread, the CLI working group is working on a solution for this space, and hearing about your use-case is helpful. Our goal is to support man output (and other documentation) in a way that is both synchronized to the actual application and that allows easy customization without hand-writing *roff.

As primitive as *roff markup rendered in man is, --help output from pretty much anything other than a program specifically configured to use clap's coloured output option is worse.

I believe this should be fixed in the current master version of Clap, since Clap will now output non-colored text when ANSII escape codes aren't supported. #963

clap-rs / clap