search

NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
17.32k stars 13.56k forks source link

Zero Hydra Failures: 19.09 edition #68361

Closed disassembler closed 4 years ago

disassembler commented 5 years ago

Hi all! It's that time of year again! That time nixpkgs contributors look forward occuring twice a year! When we branch of the new release and open a ZHF issue to track it. Lets get started tackling all the issues.

Loris_tardigradus_tardigradus_001 Zero Hydra Failures Loris_tardigradus_tardigradus_001_flip

As of today, we have 1502 (hydra build is currently running) failing jobs. For more up-to-date information, check here!

The deal is as before, we want to get this number as close to 0 as possible before the release! If you want to help, which would be much appreciated:

  1. Pick a failing package
  2. Work out why it's failing and fix it
  3. Create a PR containing the fix against the master branch (or staging if it causes a mass rebuild) and mention this issue in your description. If it gets missed out, make sure to cc @NixOS/backports to get it backported (or make a backport PR yourself, in which you also cc the backports team).

Getting latest evaluation for http://hydra.nixos.org/jobset/nixos/release-19.09

Parsing evaluation https://hydra.nixos.org/eval/1542173

Failures:

Dependency failures:

Image originally by Dr. K.A.I. Nekaris CC BY-SA 4.0

srhb commented 5 years ago

Eval report for 1541678

craigem commented 5 years ago

Eval report for 1541808

craigem commented 4 years ago

Eval report for 1541949

Ma27 commented 4 years ago

Regarding the broken NixOS tests for the Hydra module: I can reproduce the timeout locally, so this appears to be an actual issue. Currently investigating why.

NeQuissimus commented 4 years ago

Fixed xmonad test in 2442103bb79

arianvp commented 4 years ago

FYI the formatting of the first message is kind of borked, which means not all maintainers were tagged. Not sure what's up with it

globin commented 4 years ago

There is a limit of mentions per message, it's not due to formatting

WilliButz commented 4 years ago

Eval report for 1544554

worldofpeace commented 4 years ago

I'm seeing qtwebengine 5.11 and 5.9 failing with log limit exceeded

I'm guessing a fix similar/same fix as https://github.com/NixOS/nixpkgs/pull/68434 could be used. Though I'm not sure how to confirm it works without a hydra instance.

cc @matthewbauer @d-goldin

d-goldin commented 4 years ago

@worldofpeace: Yes, it is pretty much identical to #68434 (same code, WebKit/Source/wtf/ being used). Last time I simply compiled locally and confirmed smaller log output.

worldofpeace commented 4 years ago

Thanks @d-goldin. I can produce a PR for this shortly then.

makefu commented 4 years ago

nixpkgs.devpi-server.x86_64-linux got fixed by #67984

EDIT: wupps, not fixed in staging but in master for some reason

mweinelt commented 4 years ago

nixpkgs.bees.x86_64-linux was fixed by #68994.

avdv commented 4 years ago

nixpkgs.ntopng.x86_64-linux was fixed by #69096

Ma27 commented 4 years ago

Eval report for 1544918

aanderse commented 4 years ago

Looks like bot fixed a build error :smile:

https://github.com/NixOS/nixpkgs/pull/69616

Ma27 commented 4 years ago

Eval report for 1545369

Ma27 commented 4 years ago

Eval report for 1545807

worldofpeace commented 4 years ago

Eval report for 1546349

rnhmjoj commented 4 years ago

What's the deal with go failing on i686? I tried building with pkgsCross and I can't reproduce it

jonringer commented 4 years ago

not sure why it happens, but the offending lines are

--- FAIL: TestGcSys (0.06s)
    gc_test.go:33: expected "OK\n", but got "using too much memory: 44978176 bytes\n"
FAIL
FAIL    runtime 21.763s
d-goldin commented 4 years ago

@rnhmjoj: I have been trying to figure that out for a few days now, off and on, and just couldn't get it to reproduce at all either. There is a little bit of info in https://github.com/NixOS/nixpkgs/pull/70153 and https://github.com/NixOS/nixpkgs/pull/69359 (some upstream fix that didnt really work is linked).

rnhmjoj commented 4 years ago

some upstream fix that didnt really work is linked

I have just found about that issue and hoped it could fix the issue... too bad. Given it has been reported the test is flakey, do you think we could temporarily disable it?

worldofpeace commented 4 years ago

Given it has been reported the test is flakey, do you think we could temporarily disable it?

Absolutely :+1: If the test flaky, perhaps it will be fixed in a later release or we could check if other distro's had a similar issue and resolved it. I'm thinking a flaky test being flaky obviously doesn't serve well against warning any possible integration issue.

jonringer commented 4 years ago

golang 1.13.1 was released, we could try seeing if a bump fixes it :)

https://golang.org/dl/

worldofpeace commented 4 years ago

go currently defaults to 1.12.* in the release. I'm not sure how go handles their versioning but that could be a difficult change to backport to just fix a test. If picking a certain commit didn't fix it it's likely there isn't one.

jonringer commented 4 years ago

same, @kalbasit do you have any opinions on this?

kalbasit commented 4 years ago

We shouldn't be moving to Go 1.13 as the default release just yet. Although Go1compat tells us that the software is not going to break, but there are some areas that may get affected. I recommend disabling the flaky test on the affected arch.

d-goldin commented 4 years ago

Initially I was more skeptical of disabling the test, because it might have very well been some more serious issue, but if nobody can really reproduce it well and the world is rolling with the versions just fine outside of the test-env, let's just go with it in 19.09.

Here is a proposal: https://github.com/NixOS/nixpkgs/pull/70368

worldofpeace commented 4 years ago

Eval report for 1547105

srhb commented 4 years ago

Looks like #70645 accidentally killed a bunch of the nixos.* attributes, for instance by marking kscreen as broken. @disassembler was this intentional?

lheckemann commented 4 years ago

Yes, that wasn't supposed to happen… Will fix.

maggesi commented 4 years ago

blcr removed in commit 923c449e9bf554f0bd0aa2df4d1f857e144eeaf3

worldofpeace commented 4 years ago

I believe these should be closed post-release.

lheckemann commented 4 years ago

It's usually closed at one of the end of support, or the branch-off or the release of the next version.

worldofpeace commented 4 years ago

It's usually closed at one of the end of support, or the branch-off or the release of the next version.

Ah, feel free to re-open then.

matthewbauer commented 4 years ago

Could we mark all of the remaining as "broken"?

worldofpeace commented 4 years ago

@matthewbauer I believe the RM's did this?

nixos-discourse commented 3 years ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixos-21-05-zero-hydra-failures/12954/4