search

NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
18.28k stars 14.27k forks source link

Rust 1.80.0 breaks some packages #332957

Open alyssais opened 3 months ago

alyssais commented 3 months ago

Dear maintainers,

Rust 1.80.0 contains a couple of changes that break some packages: an unfixable build regression, and a warning for unrecognized cfg values that some packages treat as an error. Most commonly, the required fix is just to update the "time" crate. I've opened PRs to fix as many regressions as I can, but I can't do all of them.

1.80.0 is currently in staging-next, so will probably be in Nixpkgs master in the next 1-2 weeks. It wasn't really feasible to hold back, because packages tend to start requiring new Rust versions very soon after release, and it also didn't make a lot of sense to keep 1.79.0 as well, because many of the affected packages haven't been touched in a year or more, so there'd be no assurance we'd be able to remove it any time soon.

So, some effort will be required from you to keep your package building with 1.80.0. Usually, the process will be like this:

As a last resort, we can patch if we need to, but especially for packages that include a Cargo.lock in Nixpkgs, this is a pain.

I'll post the affected packages in comments, because GitHub limits how many mentions an issue body can contain.

Sorry for the inconvenience. I've lost a lot of the last week to coordinating the update, collecting broken packages, etc., but hopefully by spreading out the work from here it won't take too much of anybody else's time.

alyssais commented 3 months ago
alyssais commented 3 months ago
aqrln commented 3 months ago

prisma-engines is being fixed upstream in https://github.com/prisma/prisma-engines/pull/4972 but we can also patch the lockfile in the current version in nixpkgs.

alyssais commented 3 months ago

You have 1-2 weeks, remember, so if it's being fixed upstream, you can probably wait.

doronbehar commented 3 months ago

Useful link for those who want to quote the build log error for the upstream maintainers:

https://hydra.nixos.org/jobset/nixpkgs/staging-next#tabs-jobs

pbsds commented 3 months ago

or do https://hydra.nixos.org/job/nixpkgs/staging-next/<ATTRIBUTEPATH>.x86_64-linux

teutat3s commented 3 months ago

I opened an issue in the upstream repo for garage: https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/849

drupol commented 3 months ago

Typst 0.11.1 is not compatible with Rust 1.80.0, but the current development snapshot is.

alyssais commented 3 months ago

Oh right, I forgot to write above: upgrading to an unstable version is possible, at the package maintainer's discretion — it requires a judgement call on how usable/safe the unstable version will be. But getting upstream to make a release is of course the best!

justinas commented 3 months ago

teleport_15

I am somewhat inept in navigating Hydra. Could someone help me confirm whether teleport_15 is actually broken because of the Rust bump?

The last real failure I could find using the link in https://github.com/NixOS/nixpkgs/issues/332957#issuecomment-2273131448 is this one, which seems specific to x86_64-darwin and caused by nodejs failing to compile, if I'm reading this right.

Teleport 15.3.7 does seem to be using a problematic version of the time crate.

Edit: confirmed locally that it does not build.

alyssais commented 3 months ago

I am somewhat inept in navigating Hydra. Could someone help me confirm whether teleport_15 is actually broken because of the Rust bump?

Hydra hasn't built it with 1.80 yet. If you try building it yourself on staging-next you should see that it's broken.

bendlas commented 3 months ago

It wasn't really feasible to hold back, because packages tend to start requiring new Rust versions very soon after release, and it also didn't make a lot of sense to keep 1.79.0 as well, because many of the affected packages haven't been touched in a year or more, so there'd be no assurance we'd be able to remove it any time soon.

Seems to me like that sentence there contains a wee contradiction. Either the rust updates will be trivial because all the upstreams are already on top of it XOR there is this shadow looming of 1.79 becoming abandonware that we'll have to forever carry.

Looking at the regression, it's the latter one. Indeed, with https://github.com/NixOS/nixpkgs/pull/333125 I already ran into one of the classics: With 1.80, client builds, but server doesn't without cargo update (as you predicted, @alyssais) but then after the cargo update client fails with many errors.

In this case, I was able to fix it, by building against two different lock files, but that's an ugly hack and was hard to find. Obviously, I did open an upstream issue, but if I don't find it reasonable to set a 2 week ultimatium within nixpkgs, then I find it even less reasonable to impose such limits onto our upstream developers. We are not in a corporate environment here.

On principle, rust 1.80 is a new language due to the incompatible change (however inadvertent), and should be treated as such. So I think we need to leave 1.79 in nixpkgs, a little while longer. We can, however, disable its hydra builds, such that downstream will learn about the issue through increased build times and have a chance to step up, before their toys break.

bryango commented 3 months ago

Note that a simple list of patches won't work on bumping Cargo.lock dependencies as cargo vendoring happens before the patchPhase. This is quite surprising to me, but it is the documented behavior: https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/rust.section.md. A possible workaround is to use applyPatches to patch src up front; see e.g. https://github.com/NixOS/nixpkgs/pull/333143/files.

Update: please see Sandro's comment below on cargoPatches, I didn't know it exists!

SuperSandro2000 commented 3 months ago

https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/rust.section.md

buildRustPackage needs a Cargo.lock file to get all dependencies in the source code in a reproducible way. If it is missing or out-of-date one can use the cargoPatches attribute to update or add it.

alyssais commented 3 months ago

With 1.80, client builds, but server doesn't without cargo update (as you predicted, @alyssais) but then after the cargo update client fails with many errors.

This sounds like you just need to be a bit more precise with what you're updating — you don't need to update every single dependency, just the one problematic one.

alyssais commented 3 months ago

Just in case it's not obvious to anybody else, all you generally need to do is cargo update time. If you find yourself running into any sort of complication, something has probably gone wrong and you're welcome to ask for help.

errnoh commented 3 months ago

goxlr-utility should be fine after https://github.com/NixOS/nixpkgs/pull/332757 is merged. Tested with 1.81.0-beta.4.

alyssais commented 3 months ago

staging-next now has Rust 1.80.1. This fixes false positives in the dead_code lint, which IIRC broke 1-2 packages. If that's the reason your package was broken, maybe it's fine now. (But this doesn't change things for the majority of affected packages that need to update their time crate — that change isn't going away.)

errnoh commented 3 months ago

@alyssais goxlr-utility should now be compatible (assuming that you're modifying the package listing by hand)

alyssais commented 3 months ago
alyssais commented 3 months ago
alyssais commented 3 months ago
getchoo commented 3 months ago

Opened https://github.com/master-of-zen/Av1an/issues/863 for av1an and https://github.com/Spotifyd/spotifyd/pull/1297 for spotifyd. The latter should be ready for a PR here when it's merged, as we already use unstable commits

martinetd commented 3 months ago

I have no problem updating the time dependency of a package, but I don't have the time to figure out how to test it actually works (yes it's in staging-next but that doesn't have a binary cache afaik and I can't rebuild the world on my machine and 30889c3463e6 ("cargo,clippy,rustc,rustfmt: 1.79.0 -> 1.80.0") doesn't cherry-pick. EDIT: which apparently has a cache, but it's still a pretty high barrier of execution)

Having rust 1.80 optionally in tree while that mess is sorted out would be greatly appreciated so one could just change the rust build input to test (or alternatively if you have any other simple way of testing that doesn't require too much computing and storage I'll be happy to confirm...)

In my particular case:

Anyway, I'll just open a PR anyway once the build finishes with rust 1.79 in a few hours...

alyssais commented 3 months ago

yes it's in staging-next but that doesn't have a binary cache afaik

staging-next does have a binary cache. The whole point of it is that it's where we build packages in advance so they're available in master. This means that, at any given time, it probably doesn't have everything cached, but you'll very rarely be doing world rebuilds with staging-next.

ofalvai commented 3 months ago

Just opened https://github.com/NixOS/nixpkgs/pull/333637 for fixing dua

Builditluc commented 3 months ago

Created https://github.com/NixOS/nixpkgs/pull/333649 for fixing youki

gepbird commented 3 months ago

Opened https://github.com/NixOS/nixpkgs/pull/333665 for fixing cargo-geiger

martinetd commented 3 months ago

staging-next does have a binary cache. The whole point of it is that it's where we build packages in advance so they're available in master. This means that, at any given time, it probably doesn't have everything cached, but you'll very rarely be doing world rebuilds with staging-next.

Oh. I've had bad experiences with staging(not-next) in the past but I guess that was just bad timing. I could test the fix with "just" 5G of artifacts fetched from the cache, thanks.

(I stand by my argument that it could be much easier to develop/test fixes for less involved folk, but I made my point and there's no point in spamming everyone so I won't reply here any further)

emilazy commented 3 months ago

staging queues up changes for staging-next, and indeed it’s a real pain to build anything on top of staging. Whenever staging is merged into staging-next, however, Hydra works on building everything; that’s how we track regressions and make sure everything is working okay before merging into master. So as long as you’re not testing things right as a new staging-next cycle starts, staging-next should at least have stdenv and other critical build tools to test changes with.

The bandwidth use unfortunate, but not really specific to staging; testing release branch backports also uses a lot of bandwidth if you’re on unstable, and you’ll have to download all those new outputs anyway once staging-next merges into master.

(I’m not necessarily disagreeing that it could be easier to test stuff, but I’m not sure how we could accomplish it in a way that wouldn’t further increase the already too‐big workload on the people who drive wide‐scale staging changes like this.)

rgri commented 3 months ago

Opened #333793 to fix mouse-actions.

dasJ commented 3 months ago

Opened #333884 for rustdesk-flutter.

DieracDelta commented 3 months ago

grcov already has an open fix. Hopefully it is merged soon.

Mic92 commented 3 months ago

A new version of harmonia has been released, my internet is spotty, so feel free to update it in nixpkgs or otherwise wait.

a-kenji commented 3 months ago

Opened https://github.com/NixOS/nixpkgs/pull/333897 for harmonia.

cpu commented 3 months ago

blightmud @cpu blightmud-tts @cpu

https://github.com/NixOS/nixpkgs/pull/333901 will fix these two. Thanks for wrangling this alyssais!

ProducerMatt commented 3 months ago

rsbkb fixed by #333605

jnsgruk commented 3 months ago

Opened https://github.com/Screenly/cli/pull/205 (cc @vpetersson). Will follow up with a nixpkgs PR once merged.

alyssais commented 3 months ago
andresilva commented 3 months ago

polkadot should be fixed upstream in the next release (should be out in 2 weeks).

alyssais commented 3 months ago

polkadot should be fixed upstream in the next release (should be out in 2 weeks).

It's very likely staging-next will be merged before then, so if it's not okay to have it broken for a week and a bit, it probably needs a temporary fix in Nixpkgs.

Mic92 commented 3 months ago

What is nix-build-srvc?

alyssais commented 3 months ago

Sorry, that's a mistake. The package is srvc, and the maintainer is @john-shaffer.

jnsgruk commented 3 months ago

Reviews appreciated on #334390, then the screenly-cli package is all sorted! Thank you @alyssais!

alyssais commented 3 months ago
alyssais commented 3 months ago
andresilva commented 3 months ago

Opened https://github.com/NixOS/nixpkgs/pull/334414 to fix polkadot.

nh2 commented 3 months ago

I'm fixing Ceph getting broken by it in #334286

getchoo commented 3 months ago

spotifyd is fixed in #334524

alyssais commented 3 months ago

Heads up: sounds like staging-next merge is imminent, just waiting for https://github.com/NixOS/nixpkgs/pull/334473.