search

NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
17.31k stars 13.55k forks source link

Zero Hydra Failures for 17.03 #23253

Closed globin closed 7 years ago

globin commented 7 years ago

We currently have ~100 failing jobs on hydra: https://hydra.nixos.org/jobset/nixos/release-17.03

I'm hoping for a big effort from all contributors and maintainers to get this down to zero for the release at the end of march.

I'll be updating this issue with further information continuously.

Off we go on a sprint to making 17.03 the best release possible! Hydra jobset: http://hydra.nixos.org/jobset/nixos/release-17.03 Evaluation: http://hydra.nixos.org/eval/1344293

abbradar commented 7 years ago

I can't reproduce failure of nixos.tests.keymap.dvorak.x86_64-linux locally.

EDIT: seems a transient error, marked as done.

vcunat commented 7 years ago

I picked some stuff from staging that I trust. I know little about the python stuff in there, so I omitted those changes (/cc @FRidh) which made this a relatively large rebuild when compared to staging.

I created staging-17.03 branch and corresponding jobset so that we're not slowed down when fixing stuff directly on 17.03.

FRidh commented 7 years ago

@vcunat the only issue I saw was with Python 3.4 (which is hardly used in Nixpkgs) and expat. That's an easy to solve issue. The rest is fine so I think it should all go in.

the-kenny commented 7 years ago

Shall we mark packages as done when it works on hydra or when the build works locally?

globin commented 7 years ago

@the-kenny when it is commited, I regurlarly update the list in the description based on the current hydra failures.

teh commented 7 years ago

pandas on i686 is a known upstream bug: https://github.com/pandas-dev/pandas/issues/14866 - I could remove the test for that platform if that's acceptable to @FRidh ?

DerTim1 commented 7 years ago

Asterisk works locally, maybe with #23124 it would also build on hydra...

FRidh commented 7 years ago

@teh yes go ahead. Upstream doesn't test against i686. Another option is to mark it as broken for i686.

vcunat commented 7 years ago

So, all commits now merged from staging to master were cherry-picked to 17.03 directly, and that causes almost no rebuilds (all cached from staging). I just reset staging-17.03 to release-17.03, so we can still use it if need be.

teh commented 7 years ago

@peti - I sampled the Haskell failures and I'd say 80% of them are either upstream issues with e.g. newer GHCs or require manual overrides for dependencies. How about we identify the packages that people consider to be really important (e.g. I have a PR to fix purescript) and mark the rest as broken?

peti commented 7 years ago

@teh, yes, that is what I usually do. Don't worry about the Haskell packages. There will be 0 errors in a few days.

dezgeg commented 7 years ago

nixos.tests.installer.* had spurious failures - couldn't reproduce locally.

alistairbill commented 7 years ago

Can't reproduce failure of msgpack-tools on my machine or a clean VM. Ran the 'reproduce locally' script and everything compiles fine.

vcunat commented 7 years ago

@alibabzo: I see -- Downloading: libb64-1.2.1.zip in the log. Normal builds aren't allowed networking iff sandboxing is used.

magnetophon commented 7 years ago

Could someone have a look at #23394 and #23390 ? Travis has (the same?) issues on both, but I'm pretty sure at least #23394 should build.

FRidh commented 7 years ago

We have 280 failing Python packages in 17.03. https://headcounter.org/hydra/eval/352053

peti commented 7 years ago

Haskell packages now contribute 0 build or evaluation errors to http://hydra.nixos.org/jobset/nixos/release-17.03.

bachp commented 7 years ago

Regarding the failing of VirtualBox with kernel 4.10 I found that version 5.1.16 is working https://github.com/NixOS/nixpkgs/pull/23687 so maybe this needs to be backporter for 17.03.

0xABAB commented 7 years ago

Just for fun I looked at http://hydra.nixos.org/build/49548363/nixlog/3 (boomerang) and the associated source code. The C++ code is so bad that the solution to the issue would be to tell upstream to write valid C++. The reason it worked before is what I would describe as "pure chance".

(Yes, we can try to fix every problem ever created by everyone who ever touched a computer, but I don't think we should.)

My vote (not that this is a democracy) is to remove boomerang as a package from NixOS, if the author does not fix it upstream in a reasonable amount of time.

0xABAB commented 7 years ago

http://hydra.nixos.org/build/49548633/nixlog/3 is asterisk which has a dependency on some SVN repository, which is likely new, which means the dependency needs to be extracted out of the SVN repository in potentially a new package and the asterisk package needs to be updated.

It appears that asterisk is already fixed; this needs to be reflected in the status post.

0xABAB commented 7 years ago

Package 'refind' is just another case of "We can't write valid code". Such garbage can better just be directed to /dev/null.

0xABAB commented 7 years ago

http://hydra.nixos.org/build/49552500 (or one of its dependencies) should be compiled with a C++11 compatible compiler, so that's a packaging flaw for the Nix maintainer in lightdm.

0xABAB commented 7 years ago

http://hydra.nixos.org/build/49546853/nixlog/3 nbci_tools is another Nix packaging problem (it uses a compiler which is potentially too new).

0xABAB commented 7 years ago

http://hydra.nixos.org/build/49550804/nixlog/3 refers to /bin/bash. That can't possibly be healthy. It also refers to non-existing tools in its compile time configuration, which would likely make some code paths useless at run-time. I.e., it needs repackaging and possibly upstream shouldn't hardcode such stuff (perhaps they shouldn't be writing code in the first place).

obadz commented 7 years ago

Marked coreclr as broken (in 1dd16a9 & 9037001)

0xABAB commented 7 years ago

sage seems to be an upstream problem.

See: http://hydra.nixos.org/build/49667492/nixlog/1

The general trend I see is that many of the packages listed are simply of low quality. It's worth considering the question: should we encourage the distribution of sequences of bits which hardly earn the label "software"?

0xABAB commented 7 years ago

clisp requires a build time dependency on automake-1.15 and then it will probably work.

0xABAB commented 7 years ago

http://hydra.nixos.org/build/49667509 (ultrastard) specifies a dependency when the tree has the dependency itself; it probably shouldn't specify the dependency, since it fails explicitly on user-defined libraries.

0xABAB commented 7 years ago

http://hydra.nixos.org/build/49540856/nixlog/3 (octave) needs a C++11 compiler configured. So, that's a packaging bug.

0xABAB commented 7 years ago

http://hydra.nixos.org/build/49556827/nixlog/3 (panomatic) shows build tool issues (autotools) and missing symbols, which might stem from too old dependencies (the upstream build system is broken, because it shouldn't try to compile things which it cannot compile). Solution would be for upstream to fix their build system first, and then we could consider it for inclusion.

0xABAB commented 7 years ago

http://hydra.nixos.org/build/49550369/nixlog/3 (robomongo) seems so low quality that I would again vote to get rid of it completely.

0xABAB commented 7 years ago

sipp misses libpcap.

0xABAB commented 7 years ago

http://hydra.nixos.org/build/49547864 (ponyc) requires e.g. LLVM 3.9.0, but nothing newer.

0xABAB commented 7 years ago

aliceml returned this gem: 

aclocal: warning: autoconf input should be named 'configure.ac', not 'configure.in'

There are many other issues; my recommendation would be to tell upstream to fix their system first and get it out of Nix. After it's fixed, it can be repackaged. The quality is just not there to have people look at this mess.

vcunat commented 7 years ago

3.9.1 will probably be OK for ponyc, as there are only bugfixes atop.

0xABAB commented 7 years ago

@vcunat We have proof that 3.9.1 does not work in the logs.

We can tell them that ponyc's build system is broken after we have actually tried it with 3.9.0. They specifically mention 3.9.0. We should respect that, before we tell them something is wrong (when there likely isn't).

0xABAB commented 7 years ago

http://hydra.nixos.org/build/49556755/nixlog/3 (taskjuggler) requires a dependency on kde-config (no idea which package that is).

0xABAB commented 7 years ago

http://hydra.nixos.org/build/49545803 (manticore) needs upstream to fix their code: 

[[manticore.lex:208.1]] Syntax error: try inserting ;

From a packaging perspective mlton can be added as a dependency, since a WARNING is displayed when it's not found.

yorickvP commented 7 years ago

asterisk should've been fixed in #23124. Maybe pick that to the release branch?

@0xABAB I do use sage, it'd be nice if it was in nixpkgs (also, a more recent version) (currently I resolved the problem by having the sysadmins install it on an ubuntu server). The build failure seems to be related to a wrong version (also see #23304).

0xABAB commented 7 years ago

http://hydra.nixos.org/build/49548474/nixlog/3 (freestyle) checks whether the build platform is x86-64 and stops when it is not. Whether or not this should be a problem I don't know; in principle it should be possible to do cross builds, but I thought Nix didn't always do these. If cross-building works, the check could simply be removed.

Another way to fix it is to tag the build in some way such that the build infrastructure only uses x86-64 machines to build.

0xABAB commented 7 years ago

@yorickvP A way to get sage in nixpkgs is then to install it on Ubuntu and check the exact versions it is using per package and compare those to the currently specified versions in Nix and then report back on that.

7c6f434c commented 7 years ago

Tried to just pass gcc49 version of stdenv to ncbi_tools. make asked to run a specific script called reconfigure.sh in some directory, running this script fails…

doublec commented 7 years ago

aliceml returned this gem:

aclocal: warning: autoconf input should be named 'configure.ac', not 'configure.in'

There are many other issues; my recommendation would be to tell upstream to fix their system first and get it out of Nix. After it's fixed, it can be repackaged. The quality is just not there to have people look at this mess.

I was the maintainer of the aliceml package. I don't use Nix anymore and would suggest removing it if recent Nix changes have broken it.

7c6f434c commented 7 years ago

re: ponyc: I think a fresh (last week) release wants LLVM 3.9.1

vcunat commented 7 years ago

I had looked on https://github.com/ponylang/ponyc/pull/1498 and only noticed adding 3.9.1 to the list of allowed versions, without any real changes in the code. (I don't count switching CI to 3.9.1, etc.)

globin commented 7 years ago

Bumping the version fixed the build

guibou commented 7 years ago

http://hydra.nixos.org/build/49535641 (ispc) can be easily fixed by changing the flex dependency to flex_2_6_1. This is apparently a known regression of flex 2.6.3 (https://github.com/westes/flex/issues/155)

globin commented 7 years ago

@guibou fixed already

xvapx commented 7 years ago

I have looked into tribler, both the current version (6.4.3) and the latest stable release (6.5.2) depend on wxPython 2.8, wich was removed in commit 253634c4acb7f29cae84065a4ba70f2bf0ccf582. I suppose we should either bring back wxPython 2.8, mark tribler as broken, or upgrade it to a more recent beta version. I volunteer to do any of these things if you want.

vcunat commented 7 years ago

15764 claimed that wxPython 2.8 is broken. @FRidh may remember more.