Provide source tarballs with vendored dependencies?

[Charadon]

This would be extremely useful for package maintainers, as it means we don't have to package each dependency independently, also allows the program to be built completely offline, which is mandatory for many package managers due to Reproducibility concerns.

[oconnor663]

The story is different depending on which part of the project you're talking about:

• The C implementation has "no dependencies" (if C system headers don't count), so hopefully there's nothing to vendor there.
• The b3sum subdirectory doesn't explicitly vendor its dependencies, but it does check in a Cargo.lock file, so when you cargo build or cargo run you'll get a (cryptographically) pinned version of each of the dependencies. (Note that cargo install doesn't work the same way unless you add the --locked flag.) If you want to tar all of that up, you can run cargo vendor. It'll download all of those pinned deps into a vendor/ subdirectory and print out a little snippet that you can patch/append to Cargo.toml to use that directory instead of crates.io.
• The blake3 library crate doesn't commit its Cargo.lock file. This is deliberate, and it follows how Cargo does things. When a binary like b3sum depends on library crates, Cargo respects the version constraints that each library puts in its Cargo.toml file (usually implied SemVer), but Cargo does not respect library lockfiles. Only the top-level application's lockfile takes effect. The general idea is that if my library depends on your library, but your library is updated more frequently than mine, I shouldn't be preventing my callers from using a more recent version of you unless you've made it clear (through a SemVer major version bump) that your recent versions are incompatible.

Could you tell me more about what your distro is packaging and what different approaches you're considering?

[Charadon]

The story is different depending on which part of the project you're talking about:

* The C implementation has "no dependencies" (if C system headers don't count), so hopefully there's nothing to vendor there.

* The `b3sum` subdirectory doesn't explicitly vendor its dependencies, but it does check in a `Cargo.lock` file, so when you `cargo build` or `cargo run` you'll get a (cryptographically) pinned version of each of the dependencies. (Note that `cargo install` doesn't work the same way unless you add the `--locked` flag.) If you want to tar all of that up, you can run `cargo vendor`. It'll download all of those pinned deps into a `vendor/` subdirectory and print out a little snippet that you can patch/append to `Cargo.toml` to use that directory instead of crates.io.

* The `blake3` library crate doesn't commit its `Cargo.lock` file. This is deliberate, and it follows how Cargo does things. When a binary like `b3sum` depends on library crates, Cargo respects the version constraints that each library puts in its `Cargo.toml` file (usually implied SemVer), but Cargo does _not_ respect library lockfiles. Only the top-level application's lockfile takes effect. The general idea is that if my library depends on your library, but your library is updated more frequently than mine, I shouldn't be preventing my callers from using a more recent version of you unless you've made it clear (through a SemVer major version bump) that your recent versions are incompatible.

Could you tell me more about what your distro is packaging and what different approaches you're considering?

In Slackbuilds, we've been debating how to go about packaging rust programs. So far there are 3 solutions we've come up with:

1. Download each cargo file individually before beginning the build. This means we have to download and checksum, and extract each cargo file individually. Not ideal, and extremely taxing.
2. Create 2 tarballs: One is the source code, and the other tarball contains all the dependencies that are vendored. This of course means that we have to trust the maintainer of the Slackbuild with not doing some crazy stuff in the vendor tarball. ( You can even see this method used in OpenSUSE https://build.opensuse.org/package/show/security:privacy/b3sum )
3. Get a source tarball with all the vendored stuff done already, preferably from upstream. Right now, we'd have to create it ourselves.

All of these methods except the last one result in a fragmented way of doing this, and puts unnecessary trust into the maintainer of the Slackbuild. This also doesn't just apply to Slackbuilds though, it also applies to other projects too (Such as opensuse above). Overrall, we'd like to avoid the nonsense that Debian and Fedora do where they attempt to package every single cargo package.

If you guys provided a vendored tarball, it would prevent this whole Web of Trust issue =)

[musicinmybrain]

Overrall, we'd like to avoid the nonsense that Debian and Fedora do where they attempt to package every single cargo package.

As the maintainer of the rust-b3sum package in Fedora, this is exactly what we do, although opinions (in general and within Fedora) vary on whether it is nonsense. It can be tedious to have to package every crate library, but it means that a lot of work auditing crates for missing license files and impermissible content, backporting patches, and so on, can be done once rather than repeated for each package that bundles a crate. Each approach has significant advantages and significant disadvantages.

Anyway, we wouldn’t use such a vendored tarball in Fedora, but it wouldn’t affect us. We package from the crates released on https://crates.io/crates/b3sum, which I think still would not contain vendored dependencies. If vendored dependencies did appear in released crates, we would just remove them. The only thing that would be a serious annoyance would be if crates contained vendored dependencies with legal or license issues, forcing us to modify the crates before uploading them to our lookaside cache. That seems unlikely here, but it’s certainly happened before.