search

DeterminateSystems / riff

Riff automatically provides external dependencies for Rust projects, with support for other languages coming soon.
https://riff.sh
Mozilla Public License 2.0
486 stars 13 forks source link

Please make Riff packages also specify version requirements that conform to cargo conventions #159

Closed omac777 closed 2 years ago

omac777 commented 2 years ago

To include a rust crate into a project, the user must specify:

  1. crate name
  2. crate version requirements as described here: https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html

As examples:

So rather than specifying only the riff package name, why not also specify riff package version requirements that conform to cargo conventions as well? Here's an example proposed pattern:

[package.metadata.riff.targets.x86_64-apple-darwin]
build-inputs = [
  "darwin.apple_sdk.frameworks.CoreServices" = ">= 0.10.42, < 0.12.0",
  "darwin.apple_sdk.frameworks.Security" = ">= 0.10.42, < 0.12.0"
]

Thank you for listening.

cole-h commented 2 years ago

Sorry for taking a while to get back to you. I'd had this reply drafted for a while but forgot to actually send it...

The reason we don't support specifying the version is because we always use the latest version of nixos-unstable, which (almost) always only has one version of a package. What should we do if you depend on version =0.13.0 of xyz, and on version =1.0.0 of abc, but that version of abc is only present on a nixos-unstable that has 0.15.0 of xyz? A bit of a contrived example, but that's essentially why we don't handle versioning constraints for Nix packages.

I'm sorry to say, but we currently do not have plans to support version specification (for the moment, at least -- who knows what the future has in store).

omac777 commented 2 years ago

I do appreciate rolling release distros such as Nixos-unstable, Debian unstable, Debian testing, ArchLinux,  Fedora Rawhide, and Fedora Silverblue. With that said, let me provide an example within just the rust realm.  I have a project that is very typical, a set of cli tools sitting in a workspace, that reuse a particular in-house crate dependent on some other popular crates, clap, sql, crossbeam, flume, json, serde and the likes.  Nothing too wild in there, just customized for our particular workflow use-cases. We have at first labeled the crates to * to fetch the latest version such as you do in nixos-unstable.The problem we experienced is the different crate updates happen at different cycles and the different crate maintainers have other priorities than to simply keep up with other crate updates.  Years ago, we did a cargo update, clean and build and the project busted somewhere in an sql crate dependency.  We absolutely needed to refactor the code to reflect the sql crate changes to make the binary work again. LESSON LEARNED:  rust crate maintainers don't communicate with each other on a regular periodic basis forever.LESSON LEARNED:  c/c++ lib/binary maintainers don't communicate with each other on a regular periodic basis forever.We all co-exist in separate worlds and sometimes in separate enclaves.  We may not be able to communicate with each other on a regular periodic basis.

As a general rule of thumb, crates shouldn't change existing api signatures.  If they do, it breaks the crate ecosystem.  As a result, we're more prudent and have frozen on specific versions of crates until we're good and ready to do another cargo update and willing to invest the effort to refactor where need be, which is a risk as we don't know how much refactoring time is necessary.  It's an uncertainty we don't want to deal with or place effort in all the time.  We just want to use what we made that did work for as long as we can.  In other words we want to sit on our cash cow within investing any more effort in it as all businesses want to do. When you're young and without children and have a lot of time to throw energy at stuff all the time, yes frequent updates are appreciated.  When you have kids and you want to get to go home on time and it's a small shop with very few people to keep it running, then you keep things as they are preventing them from breaking often.  So pinned versions, specific versions within a set of crates are very welcome in our environment and context. So I would imagine within the c++ world, this kind of perspective would also be appreciated because when we do unit tests on c/C++ libs and binaries, we also want that level of confidence, that peace of mind when we pin a version of source code libraries known to be tested/known to be working, we want to stick with those and not risk adding new features at the detriment of breaking existing ones or breaking interoperability with other libs and tools which haven't been designed for those changes yet. Here's another example that happens all the time if we use the latest and greatest of everything EXCEPT NVIDIA.  The nvidia drivers don't keep up with the mainline kernel versions.  In a rolling release like arch or fedora rawhide, the first thing that blows up after a kernel update can be the nvidia drivers and it's irritating to no end.  That's why I appreciate fedora silverblue 36 so much for nvidia drivers.  Your rebase to fedora silverblue 37 will simply fail because there are no nvidia drivers for silverblue 37 out of the box so it just keeps on running on fedora silverblue 36 until you take huge amounts of effort make the nvidia drivers work on fedora silverblue 37.  So pinning down certain packages to certain versions is great and in this use-case scenario Fedora Silverblue 36 rocks.  I have fedora silverblue 37 on an amd box with amd gpu working with no issues.  I'm curious to know how you deal with nividia legacy drivers against newer mainline kernels without pinning them.  I hope somehow in my describing of the scenario you can appreciate the value of pinning without throwing it out as a capability.  If it wasn't useful, it wouldn't exist in the Rust/Cargo toolchain infrastructure.  By not providing something similar for nixos-unstable and riff, it places your tool at risk of not being adopted and developers will seek elsewhere to fulfill their personal and workplace requirements.  I wish your tool the best which is why I made the change request in the first place so that it imitates some of Rust crate version stuff within the C/C++ code repos and fetching infrastructure to help guarantee successful builds for the different users' requirements within a validated and labelled/tagged set of libs/repos pinned at different versions.

So with all this I hope this paints the picture in a way that crystallizes the value of specifying version requirements. Thank you for listening.  Cheers. On Tuesday, October 11, 2022 at 04:51:47 p.m. EDT, Cole Helbling @.***> wrote:

Sorry for taking a while to get back to you. I'd had this reply drafted for a while but forgot to actually send it...

The reason we don't support specifying the version is because we always use the latest version of nixos-unstable, which (almost) always only has one version of a package. What should we do if you depend on version =0.13.0 of xyz, and on version =1.0.0 of abc, but that version of abc is only present on a nixos-unstable that has 0.15.0 of xyz? A bit of a contrived example, but that's essentially why we don't handle versioning constraints for Nix packages.

I'm sorry to say, but we currently do not have plans to support version specification (for the moment, at least -- who knows what the future has in store).

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>