Man pages

[zuiderkwast]

I would like to build man pages from the markdown docs and I need some advice.

I know man pages are important for distro packaging, so I'm reaching out to you @jonathanspw and @lamby for advice. Should I include anyone else? I have a few ideas for building man pages for the entire Valkey documentation and I would like to know if these ideas make sense or if there are any other ways.

I'm willing to restructure the valkey-doc repo a bit to better fit the needs for man-pages and other formats, rather than just being a repo backing a website. (The website is one place where we publish docs, but I don't the website use case to make us pollute the doc repo with website-specific content, etc.)

• One man page per command. Section 3valkey. Is this a good idea?

• One man page for each of the markdown files under topics/ in this repo. Man page section 7, right?

  Many of these pages have generic names like transactions, so I'm thinking that we could prefix them with valkey- (e.g. valkey-transactions). Would that be a good idea? An alternative could be to use a section like 7valkey and no prefix on the page names. Any advice or preferences?

• One man page for each of the executables valkey-server, valkey-cli, valkey-benchmark, valkey-check-rdb, valkey-check-aof. (Section 1.)

  There are already some pages under topics/ that document the binaries valkey-cli, valkey-benchmark under topics. I'd like to adapt these to a man page structure used for programs and I think it would be a good idea to move them to its own directory, like programs/ or so, so topics correspond to section 7 and programs to section 1.

• One man page for the configuration. Would it make sense to call it valkey.conf? (Section 5)

• What man page fields should I use and how would you populate them? If there are

I'm considering using pandoc or some other tool that can convert markdown to man pages. (Any advice here?) In addition to that, I'll need to use some preprocessing of the markdown files (some python script, maybe some sed, etc.) and a dependency on valkey's code repo to fetch some metadata about commands. A Makefile with targets to build and install the man pages.

WDYT?

[zuiderkwast]

@guillemj

[Conan-Kudo]

I would recommend to use standard sections rather than making a custom one. User commands (e.g. client tools) are generally expected in section 1, while service admin tools and daemons are typically in section 8. Configuration file documentation is typically in section 5 (and the man page name makes sense). Concept documentation probably makes sense in section 7. Your idea of prefixing them makes sense.

I would be fine with pandoc, as it's well supported in the community.

[zuiderkwast]

@Conan-Kudo Thanks!

I got the idea of 3valkey from programming languages. In my /use/share/man/man3/ I see many 3perl, 3tcl, 3erl and even 3ssl. (I'm on debian though.)

In valkey we have command names like set and get that are not very unique. Calling a man page valkey-set would be a bit weird I think.

[jonathanspw]

Haven't had a chance to review this yet but @remicollet may have some insight too.

[Conan-Kudo]

Is 3valkey for the API? If so, then it could make sense...

[zuiderkwast]

Yes, 3valkey for the commands. That's database queries in other words.

[guillemj]

• One man page per command. Section 3valkey. Is this a good idea?

This can be problematic, but if you want to supply manual pages for the protocol commands then this is better than not namespacing the section. One problem is that sections that are not listed in for example man-db sub-section order take precedence over the others. See https://gitlab.com/man-db/man-db/-/blob/main/src/man_db.conf.in?ref_type=heads#L102 (and for example the default value in Debian is 1 n l 8 3 0 2 3type 3posix 3pm 3perl 3am 5 4 9 6 7. See https://lore.kernel.org/linux-man/Zfq1X2Fj_uqzfLqu@debian/T/#mc85c260c3cfad4499f65adfd9a4369b32580254d for a recent discussion about this same issue.

I guess the other option is to use something along the lines of valkey-proto-<command>.3 (I assume using resp- might not be desired in this context if there's an intent to distance from redis?), which might feel less ergonomic, but if you have to end up typing anyway man 3valkey <command> anyway (which currently does not bash-autocomplete, but for the former it would), then perhaps that's not actually better.

• One man page for each of the markdown files under topics/ in this repo. Man page section 7, right? Many of these pages have generic names like transactions, so I'm thinking that we could prefix them with valkey- (e.g. valkey-transactions). Would that be a good idea? An alternative could be to use a section like 7valkey and no prefix on the page names. Any advice or preferences?

Section 7 seems good to me. I think prefixing them here makes more sense, than namespacing the section. Also because with some man implementations (man-db at least) you could use "man valkey transactions" and get valkey-transactions.7.

• One man page for each of the executables valkey-server, valkey-cli, valkey-benchmark, valkey-check-rdb, valkey-check-aof. (Section 1.) There are already some pages under topics/ that document the binaries valkey-cli, valkey-benchmark under topics. I'd like to adapt these to a man page structure used for programs and I think it would be a good idea to move them to its own directory, like programs/ or so, so topics correspond to section 7 and programs to section 1.

I think providing the man pages for the programs alongside the implementation themselves would be better, otherwise you need to install an additional docs package to get these, which are unlike protocol commands or other development documentation that might be fine as independent packages.

• One man page for the configuration. Would it make sense to call it valkey.conf? (Section 5)

Yes, this is standard practice I'd say.

• What man page fields should I use and how would you populate them? If there are

Depends on the section used, you can get some inspiration from man man-pages (from the Linux man-pages project) .

I'm considering using pandoc or some other tool that can convert markdown to man pages. (Any advice here?) In addition to that, I'll need to use some preprocessing of the markdown files (some python script, maybe some sed, etc.) and a dependency on valkey's code repo to fetch some metadata about commands. A Makefile with targets to build and install the man pages.

pandoc seems fine. Depending on the massaging needed of the original, perhaps adapting the original source instead might be another option. For the code dependency, see my other comment above.

[zuiderkwast]

One problem is that sections that are not listed in for example man-db sub-section order take precedence over the others.

@guillemj Are you saying 3valkey takes precedence over 3?

I find this 3xxxx scheme pretty common and to me it seems that 3 takes precedence over 3xxxx. For example, I have zlib.3.gz, zlib.3erl.gz and zlib.3tcl installed. When type man zlib I get ZLIB(3).

This scheme 3xxxx seems common enough for interpreted languages and such things, and most non-trivial Valkey commands are pretty unique, like xreadgroup or zunionstore, which is what you need the manual for anyway, not get and set. Thus, I'm leaning towards using 3valkey.

I think providing the man pages for the programs alongside the implementation themselves would be better

Right, but we already have some docs of these in our doc repo which is under a CC-SA license, so I don't think we can move it to the code repo.

An alternative would be to include minimal man pages for the programs (identical to the output of --help) and include these in the code repo. I'm not sure how useful that would be though.

Can you as a packager include some content from our doc repo (man pages for programs) in a binary package? That would solve this issue, if it's acceptable.

Depending on the massaging needed of the original, perhaps adapting the original source instead might be another option. For the code dependency, see my other comment above.

I'll massage the original too. But the situation is that some command metadata like usage syntax is in the code repo while the bulk of the text is in the doc repo. We want to avoid duplicating the metadata in both repos.

[guillemj]

One problem is that sections that are not listed in for example man-db sub-section order take precedence over the others.

@guillemj Are you saying 3valkey takes precedence over 3?

Ah, no, sorry, my memory played tricks on me. This was a problem between two sub-sections. For 3 vs 3valkey this should be fine, indeed.

I think providing the man pages for the programs alongside the implementation themselves would be better

Right, but we already have some docs of these in our doc repo which is under a CC-SA license, so I don't think we can move it to the code repo.

Hmm, that'd be a bit unfortunate I think.

An alternative would be to include minimal man pages for the programs (identical to the output of --help) and include these in the code repo. I'm not sure how useful that would be though.

While not ideal, I think that's better than nothing. It allows to check manual pages w/o needing to run a program, or checking them online on a place such as https://manpages.debian.org/redis-cli. Perhaps they could then be filled in by writing additional content through some clean-room process?

Can you as a packager include some content from our doc repo (man pages for programs) in a binary package? That would solve this issue, if it's acceptable.

Someone packaging valkey, could include additional content such as manual pages, it would be akin to carrying a patch against upstream, but in this case just shipping the entire file. The problem is that then tracking any possible modification from the original becomes cumbersome and this then tends to become a distro-specific change. For example the Debian redis package includes manual pages written from scratch (see https://salsa.debian.org/lamby/pkg-redis/-/blob/debian/sid/debian/redis-server.1?ref_type=heads). Perhaps those could be used as a starting point instead?

[zuiderkwast]

While not ideal, I think that's better than nothing. It allows to check manual pages w/o needing to run a program, or checking them online on a place such as https://manpages.debian.org/redis-cli. Perhaps they could then be filled in by writing additional content through some clean-room process?

Good to know. Then I'll base it on the --help content with some polishing, which is pretty good already. I don't think we need the clean-room process. I'll include some "see also" to the more in-depth tutorial.

The problem is that then tracking any possible modification from the original becomes cumbersome and this then tends to become a distro-specific change.

Ok, I won't go that way then.

For example the Debian redis package includes manual pages written from scratch (see https://salsa.debian.org/lamby/pkg-redis/-/blob/debian/sid/debian/redis-server.1?ref_type=heads). Perhaps those could be used as a starting point instead?

Interesting that Chris Lamb wrote a man redis-server.1. I learned in this thread that daemons like that should be in section 8, so I'll probably go for valkey-server.8.

Everything is crystal clear now. :gem: Thanks for all the advices!

[guillemj]

Interesting that Chris Lamb wrote a man redis-server.1. I learned in this thread that daemons like that should be in section 8, so I'll probably go for valkey-server.8.

In general section 8 for servers is good advice, but that then needs to match the installation directory. In this particular case section 1 is being used because the redis-server executable is getting installed as /usr/bin/redis-server instead of under /usr/sbin. But for valkey perhaps you could consider to install by default under /usr/libexec/valkey/<server-name>, then section 8 also fits the bill, and it does not clutter bin nor sbin directories? :)

Everything is crystal clear now. 💎 Thanks for all the advices!

Any time! Always glad to assist upstreams making downstream work easier. :D

[zuiderkwast]

/usr/sbin/  Non-essential system binaries, e.g. daemons for various network-services.

Thanks again for more enlightenment. Makes sense. I wonder if changing can cause trouble though. We already have a release installing valkey-server in /user/bin. We have some tools that are just symlinks to valkey-server which checks how it was invoked. This includes valkey-check-rdb for checking dump files. It's definitely easier to keep all this in /usr/bin if that's acceptable.

[Conan-Kudo]

There's no real reason to move it from /usr/bin to /usr/sbin, so I wouldn't worry about it. There are plenty of things in /usr/bin that use section 8 for man pages.

For the issue of having the man pages come out of this repo, as long as we have synchronized tags, it's fine for @jonathanspw and I, as we can just pull two tarballs in our spec file.

[zuiderkwast]

@Conan-Kudo I can add some variable to tweak the man page section (1 or 8) and leave the decision to you. :)

We don't have synchronized doc tags. We don't have doc tags at all actually. No versioned docs.

We could consider tagging versions in the docs repo though, if you think it's important.

[Conan-Kudo]

If you want to have the man pages come out of this repo, then it would make sense that this repo gets tagged the same time the code is with matching tags. That way it's easy for us to pull it in.