Add man page (tokei.1)

[gyscos]

Having a man page would be a great addition to the current help message.

Clap can't quite generate it yet (https://github.com/kbknapp/clap-rs/issues/552). In the meantime, other projects like ripgrep use pandoc to create the manpage from markdown.

[XAMPPRocky]

Thank you for this issue! This isn't really a high priority at the moment so I'm comfortable waiting on clap to provide that functionality.

[ntBre]

Is there any interest in incorporating clap_mangen? I'd be willing to submit a PR if so. I just tried to run man tokei today.

[XAMPPRocky]

@ntBre Thank you for your interest! Yes I would be, however the thing that needs to be figured out is, how to incorporate manpage generation into the distribution, is it something we distribute in the release? do we have a command to manually install it? Things like that.

I’d like to hear answers to those questions first, with examples from other rust projects.

[ntBre]

Thanks for the quick response! I guess I'm a bit biased toward Arch Linux and haven't thought much about how Rust projects are installed otherwise. For the paru AUR helper the manpage is included in the distribution, and installation is handled by the PKGBUILD.

ripgrep appears to be the same (man page in distribution, installed by PKGBUILD). And so does fd. I'm assuming the process is similar for other distros.

Because clap_mangen appears to want to run in the build script, manpage generation will happen as part of the normal build process. So I think the only change to the release-generating CI would be to add the resulting man directory and page to the tar files. That's assuming I'm reading and understanding the .github files correctly, though.

As an alternative, I'd also be happy to write (or copy one of these old PRs as a starting point) a man page by hand, if you think that the maintenance burden would be lower now that the project is more mature.

Just as a quick summary response to your questions, in my opinion of course,

• manpage generation runs in build.rs as part of creating a release
• yes we distribute it in the release
• no command to manually install it

We could add a line in the README or something if we really want to tell people a way to install it manually, but these seem to be in line with the other Rust projects I looked at too.

Again, thanks for the help. I love using tokei!

[drainpixie]

Any update on this?

[XAMPPRocky]

yes we distribute it in the release

Right, but tokei is distributed in a variety of ways, and including it in the release isn't going to work for cargo install for example, or how would it work for homebrew and other package managers? Distributing it in the release is good enough for the GitHub releases but that's not the most common way people install tokei.

[ntBre]

So we shouldn't have something that some people can use if everyone can't use it? Writing my own man page locally is good enough for me. I don't have to contribute it here if you don't want it. It seems like multiple people have wanted it in the past, though.

I did my best to look into the distribution mechanisms I'm familiar with and use, but I can't speak for people who cargo install or use homebrew. They're certainly not going to get a man page from the current distribution, one way or another.

Maybe I'm misinterpreting your comments, but to me this should be a clear "yes we should have a man page in the repository" and anyone who wants to package/use/distribute it can figure that out (and contribute it back) later. I'm not really sure why this issue has been open for 5.5 years.

[XAMPPRocky]

So we shouldn't have something that some people can use if everyone can't use it?

I didn't say that, but when I'm maintaining a project I have to weigh the maintenance, and if I'm going to have maintain something for the lifetime of the project I want to make sure it highest reach to users as possible to provide a consistent experience. Especially when your proposed implementation would exclude the highest percentage of my users that heavily skews against accepting that proposed implementation. I'd prefer to include it as a command in the tool because that is something that is compatible with all distributions.

We should have a manpage but I don't want it to have any extra burden of maintenance because it is a niche nice to have, not a core feature of the project. Which is why I've not accepted manually written manpages and have only been interested in supporting it with projects like clap_mangen.

I'm not really sure why this issue has been open for 5.5 years.

Because it's a personal project I make and maintain in my spare time ontop of dozens of other projects and a full time job. I can understand for things not going at a pace you would like, but this project isn't a full-time job for me, it operates at the behest of my time and preferences, you're free to fork it if you'd like to operate it differently.

[ntBre]

Alright, I understand it's a personal project and not your main priority, of course. I just mean that you've had multiple people offer to contribute the whole man page in the past 5.5 years.

I still don't fully buy the "maintenance burden" argument because the man page only really has to change when the clap options change. A quick scroll through the src/cli.rs file shows the addition of the --languages flag 2 months ago, but the last change before that (besides updating dependencies 18 hours ago) was 2 years ago, and I didn't even check if that actually changed the flags. I did agree that the burden was high when the project was new, but I don't think you're frequently adding/renaming/removing flags now that the project is mature. I could be wrong about that, though. And if someone else makes a PR adding a flag, they should have to add it to the man page too.

As for the distribution scheme, could you expand a little more on how the "highest percentage of [your] users" install this package? I've only ever installed it on Arch with pacman -S tokei. I assume that every Linux distribution has a clear way of including man pages along with programs they install just like Arch does. I tried checking some of the other ones in the README, but they don't expose things as clearly as Arch PKGBUILDs do. Homebrew at least mentions installing man pages as just copying files to the right place. Whoever packages tokei for a distribution likely just needs to add a line or two to whatever build/install script they use, whether it's a Homebrew formula, an APK from Alpine Linux, or whatever.

I'm assuming you're not an Arch user, but the official package for tokei is this file:

# Maintainer: Sven-Hendrik Haase <svenstaro@gmail.com>
# Maintainer: Orhun Parmaksız <orhun@archlinux.org>

pkgname=tokei
pkgver=12.1.2
pkgrel=1
pkgdesc='A blazingly fast CLOC (Count Lines Of Code) program'
arch=('x86_64')
url="https://github.com/XAMPPRocky/tokei"
license=('MIT' 'Apache')
depends=('gcc-libs')
makedepends=('rust' 'cargo')
source=("$pkgname-$pkgver.tar.gz::https://github.com/XAMPPRocky/tokei/archive/v${pkgver}.tar.gz")
sha512sums=('b8474cb3cad8cab8cb9c24b44a9b7bdaa436fde4e56ca25a8c6d9cbe342b27acf8041cda15da5e74b4fed5291b3eacd5b298f569e0c8041e9146f89847fe47ce')

build() {
  cd "${srcdir}/${pkgname}-${pkgver}"
  cargo build --release --locked --features all
}

check() {
  cd "${srcdir}/${pkgname}-${pkgver}"
  cargo test --release --locked --features all
}

package() {
  cd "${srcdir}/${pkgname}-${pkgver}"

  install -Dm755 target/release/tokei "${pkgdir}/usr/bin/tokei"

  install -Dm644 LICENCE-MIT "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE-MIT"
  install -Dm644 LICENCE-APACHE "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE-APACHE"
}

They just retrieve the package from the GitHub releases, unpack it, and install. I assume every other package manager works the same way. Again, I could be wrong and would accept that if you know better.

Anyway, the burden of updating all of those distributions falls on the distro maintainers themselves, unless you've been writing all of the build scripts like this for all of the distributions listed in the README. That only leaves cargo install users. If they're the majority, they're just out of luck because rust-lang/cargo#2729 has been open since 2016. As a slight aside, I'm also curious how you know how your users install the package.

This is not my full time job either (although it may be hard to tell today), and I just wanted to contribute to a project I use and enjoy. Again, my own issue will be solved by writing my own man page and installing it on my machine. I just thought other people could benefit from that work too. I've already made my fork, I'll just do it in there and send a PR that you can do with as you will.

[willeccles]

The additional maintenance overhead is trivial; simply add a couple lines to the CI task that generates releases to add the man page (and of course, add whatever is required to the build process such that it generates the man page, if necessary). Since package managers do not use these generated artifacts and will build the software from source, this doesn't affect package repositories--they will need to update their package scripts. In the example arch package above, they simply must add the man page to the installation artifacts. If I am misunderstanding and it's not added as part of the standard build and requires a second build command, then that will need to be added as well, which is no big deal. Generating code and documentation is a common and expected part of building even mildly complex software, and it should not be difficult to add man page generation to the build process and let package maintainers adapt to install the man page.

I would argue that even if the majority were using cargo install (which I don't have data on), they are misusing their distribution's package management tools, which is their decision to make. However, it is expected (hence the existence of this issue) that components of a Unix-like system have manual pages, rather than being a "niche nice-to-have." Given that the tooling already used here can generate the man page for you and it incurs nearly zero maintenance overhead, why not do it? Someone else can do this in a PR (I would, but I am not a rust developer myself--if nobody else is willing to, however, I can do it), and you don't really have to maintain much of anything after it's done once. I do understand that this is a side project for you, and it's not like this is an urgent need, so it doesn't have to be done ASAP.

TL;DR: Generating it as part of the build process and including it in GitHub artifacts seems to be trivial. Package maintainers are already used to installing manual pages as part of their packages, so it will be a minor change for them. Minimal maintenance overhead results from doing this. I suggest adding man page generation to the build process for tokei.

[willeccles]

Despite not being a rust developer, I just investigated how clap_mangen may be used for Tokei. It looks like the only thing blocking the ability to do it with the existing setup is that there's no way for build.rs to get access to the commands defined in src/cli.rs. I found another project which uses a different setup for defining commands. The CLI is defined at the top level of the source file in this project. Then, the build.rs includes this file and can use clap_mangen to generate a man page from it. With Tokei's current setup, the clap "App" is defined in a function as a local variable. I think if this was defined at the top level of the Cli struct (or otherwise), a similar approach could be taken in Tokei's build.rs to generate the man page. Unfortunately, as I am not a rust developer, it would take me quite some time to sort this out myself, but someone who is more familiar with rust could probably make this work.

I suppose it's also possible that you could just duplicate the whole clap command definition in build.rs, but that most definitely falls into the category of "maintenance nightmare."

[ntBre]

@willeccles As a Rust developer, I'd definitely be willing to work on this, especially with the great head start you've outlined. I'm just waiting on the go ahead from the maintainer. For my own use, I'm still planning to write a man page manually but haven't gotten around to it yet.

[willeccles]

@ntBre If you want to save yourself the effort, you can also just add clap_mangen as a runtime dependency and make a hacky build that quickly generates a man page in src/cli.rs. Not ideal, but certainly easier than writing your own man page!

[ntBre]

A little bit of roff never hurt anyone! Especially if you copy an existing one and do a little find and replace on the output of tokei -h. But I will give clap_mangen a try, you've piqued my curiosity again for sure.