Support installing manpages (and potentially other files)

[joshtriplett]

I use Cargo to build a command-line tool written in Rust. I've also written an accompanying manpage, command.1. I'd like to tell Cargo about manpages via Cargo.toml, so that cargo install will install them to the appropriate location (gzip-compressed and installed to ${root}/share/man/man1/command.1.gz, where the section 1 in man1 gets extracted from the first character of the manpage extension).

Note that once Cargo has a mechanism for a post-build script, some packages may wish to use that to build the manpage. For instance, a package might parse --help output to generate a manpage, process some other document language to produce a manpage, or use help2man-like functionality from an argument parser. However, for a first pass, I'd just like to have support for installing manpages already written directly in manpage format.

[joshtriplett]

Thinking about this, I think that with the specification hashed out, this becomes an E-easy issue. Given that, I'd like to propose a more precise specification for how Cargo should handle this:

cargo install should automatically detect manpages in a man subdirectory adjacent to Cargo.toml. Manpages should match the glob man/*.[1-9]. Given a manpage man/command.N, cargo install should compress it with gzip (with no timestamp and no embedded filename, as though with gzip -9n), and install it to share/man/manN/command.N.gz relative to the install root.

Optionally, cargo install could gain a new option --no-man to disable installing manpages.

(I think it makes sense to use convention-over-configuration for this. If someone has a use case for specifying additional manpages by filename or directory name, we can always add a new field to Cargo.toml at that point.)

[davidszotten]

just to note that bash (or other shell) completion might be another reasonable "potential other file" for a binary to ship

[alexcrichton]

@joshtriplett yeah I agree that once we have a specification for this it's an easy bug to knock off, but unfortunately the specification is the hard part. I don't think we'll want to extend Cargo to install just man pages (if at all) as as @davidszotten mentioned there's other things like bash completions, perhaps documentation, and even files like assets in games.

We don't want Cargo to become a replacement package manager for the system or other "proper" installation methods, so I'm hesitant to make progress on this as well. The purpose of cargo install was a cross-platform method of quickly sharing code with others using Rust.

This probably wants to be discussed a little more broadly before trying to tackle each of the sub-issues associated.

[joshtriplett]

@alexcrichton I can understand that, but on the other hand, cargo doesn't quite yet support the constraints of distro package managers either, though folks are working on that. And in any case, distribution package managers don't support installing into $HOME, while cargo does.

I definitely wouldn't want to use cargo to install anything systemwide into /usr; however, man will also automatically find manpages installed into ~/.local/share/man/manN/command.N , so I'd like to have this support in cargo for installing into $HOME.

For that matter, many distribution package build systems just have the package install itself into a temporary directory via $DESTDIR, and then tweak and package up that temporary directory. So, if cargo knows about various assets to install, the distribution package build system can just use that.

[alexcrichton]

Yeah right now we just don't want to open the floodgates to a whole litany of various post-installation steps of Cargo just yet. It may be worth prototyping this as a new subcommand (perhaps leveraging package.metadata to store configuration) before moving into Cargo as well.

[vu3rdd]

We could create a .cargo/share/man directory where the cargo install can install the manpage into, depending on the section (inferred from the name of the man page - foo.N). And then, as part of the installation of rust/cargo toolchain, we could add .cargo/share/man into the MANPATH environment variable.

How does that sound?

[jugglerchris]

I have a use case where I want to install extra non-manpage files, but need the binary to know the path to them somehow (ie when installed needs to point to the install location, otherwise to somewhere in the source tree). I might be up for prototyping a cargo subcommand.

[joshtriplett]

We could create a .cargo/share/man directory where the cargo install can install the manpage into, depending on the section (inferred from the name of the man page - foo.N). And then, as part of the installation of rust/cargo toolchain, we could add .cargo/share/man into the MANPATH environment variable.

You don't actually need to add $TARGET/share/man to $MANPATH if $PATH already contains $TARGET/bin; man will automatically look for ../share/man relative to directories on your $PATH.

But otherwise, cargo install installing manpages named foo.N into $TARGET/share/man/manN/ sounds perfect.

[cardoe]

I'd like to see this relative to the --root argument to cargo install so in the default case it goes to $HOME/.cargo/share/man so that it just works for people that have $HOME/.cargo/bin in their path. But that's cause I'm lazy and use cargo install --root=${D}/usr && rm /usr/.crates.toml for packaging.

[casey]

I propose that we tackle this issue in two steps:

1. Decide on a standard way for crate authors to ship man pages with their crate. I think that man/*.[1-9] in the crate root would be a very reasonable way to go.

2. Later decide if we want to extend cargo to actually install man pages and the details of how that would work.

I think that part 1 would be easy and uncontroversial, since there isn't any code to write or behavior to decide on. Also, it would would be very useful on its own for downstream packagers of rust binaries. For example, tools like https://github.com/mmstick/cargo-deb could be extended to include man pages in the .deb, if present.

[joshtriplett]

@casey That sounds good to me. I'd be happy to enable that in debcargo as well.

[cardoe]

On 2/10/17 10:04 PM, Casey Rodarmor wrote:

I propose that we tackle this issue in two steps:

1.

Decide on a standard way for crate authors to ship man pages as part
of their project. I think that |man/*.[1-9]| in the crate root would
a very reasonable way to go.

2.

Later decide if we want to extend cargo to actually install man
pages and the details of how that would work.

I think that part 1 would be easy and uncontroversial, since there isn't any code to write or behavior to decide on. Also, it would would be very useful on its own for downstream packagers of rust binaries. For example, tools like https://github.com/mmstick/cargo-deb could be extended to include man pages in the |.deb|, if present.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rust-lang/cargo/issues/2729#issuecomment-279119341, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAih5WT_A-xnOqPpDHk0_k6xfXpgspUks5rbTM4gaJpZM4IkmVI.

FWIW, Gentoo and Yocto already do this with man pages appearing in man/*.[1-9] because that's how I ship my crates.

-- Doug Goldstein

[alexreg]

If we could extend this beyond man pages to general data files, that would be great.

[lilyball]

I think manpages are the most useful thing to handle first, and should be tackled before handling arbitrary other files.

Also, while a man/foo.N convention to auto-detect manpages is fine, Cargo.toml will need a way to explicitly specify the location for manpages as well, as there are other conventions in use (e.g. oghamn/exa uses contrib/man/exa.1, and I've seen share/man/man1/foo.1 in other projects).

[joshtriplett]

@kballard That seems reasonable. Convention first, but also a Cargo.toml key that accepts either a string or list of strings; the strings can specify either directory names (install all manpages in the directory) or filenames (install the specified manpage).

[BartMassey]

In my opinion, once we've gone down the road of having cargo install and a $HOME/.cargo, it is unreasonable not to be able to have the install put arbitrary needed assets in there. I would hate to see this limited to just manpages. The distro package manager is not really a viable substitute for one's own programs under development and in use on one's own machines...

[alexreg]

I fully concur with @BartMassey on the above.

[hasufell]

This is a must-have feature. You might also want to check how cabal handles this:

• in the package definition there is a data-files setting
• you can get the data locations via generated module functions in the code, so it doesn't really matter if it's installed into $HOME, still residing in the build dir or already system-wide (if applicable)

Automagic detection of manpages is fishy, imo. Instead, there should be wildcard support in the corresponding config syntax.

Also, doing this properly and increasing compatibility with package managers will also involve having more fine-grained control over the installation destination (not just --root), because otherwise it will make crate devs hardcode weird and incompatible suffix directories.

I wouldn't want to give full control over the installation procedure to the crate dev. It should not be possible to define arbitrary installation destinations. There must be a set of things, most likely corresponding to the FHS, like --mandir and --datadir, which can be extended carefully, if needed. But this format must be decided upon first in a generic way. As such, I agree with @BartMassey ...it will just make it worse if this is handled per use case.

Or to put it simple: this feature is not a substitute for package managers at all, instead it increases package manager compatibility (if done right).

[alexreg]

@alexcrichton Could we kindly get your thoughts on the recent developments in this discussion? 😊

[alexcrichton]

Adding support for this was explicitly avoided in the initial RFC for this feature. To me that means that adding support will likely entail a further RFC with rationale, detailed design, etc.

[alexreg]

@alexcrichton I didn't see any rationale given there, however. Is there? This seems to be a much in-demand feature... perhaps someone here (more qualified than me) could take up the task of writing an RFC though.

[zeenix]

@alex reading through that link your provided, it sounds to me that installation and uninstallation of data is considered to be only useful for deployments/distros and not devs. As a maintainer/developer of system D-Bus services (Geoclue and gps-share), I need to install (and sometimes also uninstall) files from the source/build directory to test changes etc. It would be very annoying for folks like myself to have to create custom scripts or rpm packages for just being able to install files.

[remexre]

I think that installing to a $DESTDIR (which just so happens to default to ~/.cargo) should be the priority -- creating actual distro packages de facto requires this. I think @hasufell has the right idea, maybe have something like:

# globs supported for all paths
# paths are relative to workspace root directory

[assets]
# defaults to ["man/*.[0-9]"]
# install to mandir
manpages = ["example.1"]

# assets.data defaults to []
# assets.data.FOO = { path = "BAR" } installs BAR to datadir/FOO
[assets.data."icon.png"]
path = "img/icon.png"

[joshtriplett]

Checking back about the current status of this.

I brought this up in the @rust-lang/cargo meeting, and there wasn't any objection to the premise, just a concern about getting the details nailed down.

Is anyone interested in writing down an RFC for this? I'd be happy to talk to someone about possibilities and requirements, for both autodetection and manual specification.

[joshtriplett]

Please see https://github.com/rust-lang/rfcs/2376 for an RFC proposing a solution to this.

[alexreg]

@joshtriplett Link doesn't work unfortunately.

[lnicola]

https://github.com/rust-lang/rfcs/pull/2376

[michaelmior]

For a stopgap I thought I would share a script that I use to link man pages from crates in my local registry. It doesn't always work (e.g. when the man page is templated and generated at build-time) but it does a decent job.

[jmmv]

Interested in this feature as well. Something that hasn't been mentioned so far, I think, is that it's very common for any installation procedure to also install "package documentation" (those top-level files in capital letters, like README, NEWS, etc.) under share/doc/<package>/. Just wanted to make sure that it's possible to put these in place as well with whatever mechanism is implemented here.

[alexreg]

@jmmv Yeah. Ideally it would be possible to install anything (in a declarative way) under /usr/local/ or similar. I really think we need to move forward on this. Would anyone be interested in writing up an RFC with me on this topic?

[lilyball]

We don't need to install in /usr/local/ or similar. The default behavior for man if you don't specify a MANPATH is to build one based on PATH, so we can just install manpages into ~/.cargo/share/man and man will pick it up automatically as long as MANPATH isn't set.

[jmmv]

@kballard That's not a standard feature. From the reads of it, it's a GNUism (doesn't work on macOS, for example), and the MANPATH derivation from PATH depends on the mappings configured in /etc/manpath.config.

[michaelmior]

Personally, I would still rather have man pages installed into ~/.cargo and then have to set my MANPATH manually instead of having cargo install mess with /usr/local.

[lilyball]

It does in fact work on macOS. In fact, the language describing this in macOS's manpage is identical to the one on a Linux manpage.

If it doesn't work on macOS it's probably because you have MANPATH set.

[jmmv]

Uh huh. I knew I had MANPATH set when verifying this... and then forgot to unset it during my testing. Silly me. TIL.

[alexreg]

You can get around this for man pages, sure, but what about things like system fonts and other system-wide resources?

[lilyball]

Cargo isn't a package manager. I think it should install stuff into ~/.cargo that makes sense (like manpages), but definitely shouldn't be installing anything system-wide.

[remexre]

Furthermore,

<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
<dir>/home/remexre/.cargo/share/fonts</dir>
</fontconfig>

or something similar should be enough to get cargo-installed-to-share fonts working

[alexreg]

@kballard We certainly may want to adopt that view, but not necessarily. This PR is inherently all about making Cargo more like a package manager. We need to decide where to draw the line. What about, for example, installing (and removing) cron/launchd services? Let's see what people think about this first, in particular @joshtriplett and the like. I'm not of particularly strong opinion here, but it's up for debate at this time.

[joshtriplett]

My use case is primarily installing into a directory like debian/tmp/ for subsequent packaging. I don't need to install to system/privileged directories at all. I would like to be able to install arbitrary files to arbitrary sub-paths of the target directory, though.

[casey]

Almost every package manager wants to install things into a different place, and trying to teach Cargo how to install artifacts in directories outside of .cargo feels like an exercise in futility. Especially if we want Cargo-installed artifacts to uninstall cleanly.

Just anecdotally, I would prefer Cargo not to install things to /usr/local, since I manage /usr/local with homebrew, and homebrew and Cargo might not interact well.

One way that we can interact well with third party package managers is to allow crates to declare associated resources, like man pages. Then, someone interested in packaging rust crates for a given package manager just needs to write a script that reads the locations of those resources from the crate manifest, and move them to the right place. This would allow packagers to treat all rust crates uniformly, while also sparing Cargo from having to understand the intricacies of the many package managers and system layouts in existence.

Installing man pages into a subdirectory of .cargo is probably safe. A conservative approach mightbe to add a key to Cargo.toml like man-pages = "man", that declares a subdirectory that contains man pages. Then, on cargo install, those man pages can be copied to ~/.cargo/share/man. Third-party packagers can write a script that reads the manifest, and installs the contents of that directory to the appropriate location.

It's also worth considering that man pages might be generated from other sources, so allowing a crate to specify build-outputs as man pages might also be useful.

[alexreg]

@joshtriplett That sounds like a reasonable demand. Perhaps we're coming to a consensus after all...

@casey I think the idea is to have the build.rs script inform Cargo via a directive of what man page to install. It's thus still reasonably declarative, allows for uninstallation, but also supports dynamic generation.

[dkasak]

One way that we can interact well with third party package managers is to allow crates to declare associated resources, like man pages. Then, someone interested in packaging rust crates for a given package manager just needs to write a script that reads the locations of those resources from the crate manifest, and move them to the right place. This would allow packagers to treat all rust crates uniformly, while also sparing Cargo from having to understand the intricacies of the many package managers and system layouts in existence.

This is the right idea, IMO (barring any changes to support dynamic generation and whatnot). There seem to be two rather separated concerns here:

1. Teaching Cargo to know about additional resources (man pages, systemd service files, shell completions) that a program requires in a declarative manner and exposing the location and type of these resources to external scripts.
2. Teaching Cargo itself to install (i.e. copy) files to some location.

Of the two, 1 is much more important to get Rust crates packaged into distro package managers properly. It's also something that language tooling rarely gets right, unfortunately.

The way this is accomplished does not matter as much, as long as packagers can consistently find these resources at a predictable location after building the crate. Of course, one possible way of achieving this is DESTDIR-like support mentioned by remexre earlier, which also handles 2.

I don't think Cargo should try to install things system-wide itself.

[michaelmior]

+1 for not having Cargo install things system-wide. The only downside of the proposal above is that there doesn't seem to be an obvious way to allow these resources to be created as part of the build process.

[alexreg]

Can we stop rehashing the issue of Cargo making system-wide installation, please? We agreed that was a bad idea a while ago. The consensus solution seems to be to install to a sandbox in ~/.cargo. And yes, there is "an obvious way to allow these resources to be created as part of the build process" -- that's what this whole conversation has been about.

[michaelmior]

@alexreg I was referring specifically to the comment from @dkasak. It sounded like the idea was just to add path information to the crate manifest. Perhaps I misunderstood, but it's not obvious to me unfortunately how this would allow man pages to be built.

[alexreg]

@michaelmior Oh, okay. Well, I was under the impression people were gravitating towards the idea of a post-install.rs file in the style of build.rs, which can emit directives in a similar way too, including paths to man pages to be included (copied into the sandbox).

[dkasak]

Perhaps I misunderstood, but it's not obvious to me unfortunately how this would allow man pages to be built.

This is what my somewhat handwavey "barring any changes to support dynamic generation and whatnot" was meant to address. Sorry about the confusion.

The point of my comment was mostly reiterating that it's important that the location of these special files is known after the build/install is finished, so that they can be easily packaged by distro package managers. If consensus is already forming towards that, that's great news.

[alexreg]

Well, we really need an RFC at this stage to flesh out some of the idea and partial consensuses above. :-)

[tchernobog]

Cargo isn't a package manager.

Then the official description is misleading: https://doc.rust-lang.org/cargo/

Cargo is the Rust package manager.