Broken upstream tarballs in 15.09

[edolstra]

I ran copy-tarballs.pl on Nixpkgs 15.09 to mirror all referenced tarballs/patches in the content-addressed fetchurl mirror at tarballs.nixos.org. It successfully mirrored 14850 files (in addition to 3861 files mirrored previously). But it failed to mirror 83 files. There were 19 hash mismatches:

error: hash mismatch for ‘http://invisible-island.net/datafiles/release/cdk.tar.gz’
error: hash mismatch for ‘http://kernel.ubuntu.com/~cking/tarballs/stress-ng/stress-ng-0.04.15.tar.gz’
error: hash mismatch for ‘http://perso.b2b2c.ca/sarrazip/dev/boolstuff-0.1.14.tar.gz’
error: hash mismatch for ‘https://codeload.github.com/Thomas-Tsai/partclone/legacy.tar.gz/stable’
error: hash mismatch for ‘https://geolite.maxmind.com/download/geoip/database/asnum/GeoIPASNum.dat.gz’
error: hash mismatch for ‘https://geolite.maxmind.com/download/geoip/database/asnum/GeoIPASNumv6.dat.gz’
error: hash mismatch for ‘https://geolite.maxmind.com/download/geoip/database/GeoIPv6.dat.gz’
error: hash mismatch for ‘https://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.xz’
error: hash mismatch for ‘https://geolite.maxmind.com/download/geoip/database/GeoLiteCityv6-beta/GeoLiteCityv6.dat.gz’
error: hash mismatch for ‘https://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz’
error: hash mismatch for ‘https://github.com/adobe-fonts/source-code-pro/archive/2.010R-ro/1.030R-it.tar.gz’
error: hash mismatch for ‘https://github.com/namecoin/namecoin/archive/nc0.3.80.tar.gz’
error: hash mismatch for ‘https://github.com/numixproject/numix-icon-theme-circle/archive/d7e8c4cdcf.zip’
error: hash mismatch for ‘https://github.com/ocaml/ocaml/pull/117.patch’
error: hash mismatch for ‘https://github.com/valr/cbatticon/archive/1.4.2.zip’
error: hash mismatch for ‘http://voxelands.com/downloads/voxelands-1506.00-src.tar.bz2’
error: hash mismatch for ‘http://www.chnry.net/ch/IMG/zip/puremapping-libdir-generic.zip’
error: hash mismatch for ‘http://www.ladspa.org/download/ladspa_sdk_1.13.tgz’
error: hash mismatch for ‘mirror://sourceforge/ats-lang/ats-lang-anairiats-0.2.12.tgz’

And 48 files that couldn't be downloaded:

error: unable to download ‘ftp://ftp.archlinux.org/community/os/x86_64/sdlmame-0.151.u0-1-x86_64.pkg.tar.xz’: Couldn't resolve host name (6)
error: unable to download ‘ftp://ftp.au.debian.org/debian/pool/main/b/bobcat/bobcat_3.25.02.orig.tar.gz’: Remote file not found (78)
error: unable to download ‘ftp://ftp.au.debian.org/debian/pool/main/s/signing-party/signing-party_2.0.orig.tar.gz’: Remote file not found (78)
error: unable to download ‘ftp://ftp.foolabs.com/pub/xpdf/xpdf-3.02pl4.patch’: Remote file not found (78)
error: unable to download ‘ftp://ftp.mozilla.org/pub/mozilla.org/directory/perldap/releases/1.5.3/src/perl-mozldap-1.5.3.tar.gz’: Couldn't connect to server (7)
error: unable to download ‘ftp://ftp.mozilla.org/pub/mozilla.org/directory/svrcore/releases/4.0.4/src/svrcore-4.0.4.tar.bz2’: Couldn't connect to server (7)
error: unable to download ‘http://directfb.org/downloads/Core/DirectFB-1.7/DirectFB-1.7.7.tar.gz’: Couldn't resolve host name (6)
error: unable to download ‘http://dl.lm-sensors.org/i2c-tools/releases/i2c-tools-3.1.1.tar.bz2’: HTTP response code said error (22)
error: unable to download ‘http://download.ag-projects.com/MSRP/python-msrplib-0.17.0.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://download.clipgrab.de/clipgrab-3.5.1.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://downloads.sourceforge.net/project/mednafen/Mednafen-Server/0.5.2/mednafen-server-0.5.2.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://eid.belgium.be/en/binaries/eid-viewer-4.1.4-v4.1.4.src.tar_tcm406-270732.gz’: HTTP response code said error (22)
error: unable to download ‘http://forge.scilab.org/index.php/p/arpack-ng/downloads/607/get/’: HTTP response code said error (22)
error: unable to download ‘http://fortawesome.github.io/Font-Awesome/assets/font-awesome-4.4.0.zip’: HTTP response code said error (22)
error: unable to download ‘http://ftp.gwdg.de/pub/languages/perl/CPAN/authors/id/E/ET/ETHER/JSON-MaybeXS-1.002005.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://ftp.gwdg.de/pub/languages/perl/CPAN/authors/id/E/ET/ETHER/MooseX-Getopt-0.65.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://ftp.gwdg.de/pub/languages/perl/CPAN/authors/id/E/ET/ETHER/namespace-autoclean-0.20.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://ftp.gwdg.de/pub/languages/perl/CPAN/authors/id/E/ET/ETHER/Test-Warnings-0.016.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://ftp.gwdg.de/pub/languages/perl/CPAN/authors/id/E/EX/EXODIST/Test-SharedFork-0.29.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://ftp.gwdg.de/pub/languages/perl/CPAN/authors/id/J/JD/JDHEDDEN/threads-2.01.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://ftp.gwdg.de/pub/languages/perl/CPAN/authors/id/M/MI/MIROD/XML-Twig-3.44.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://ftp.gwdg.de/pub/languages/perl/CPAN/authors/id/R/RI/RIBASUSHI/DBIx-Class-0.082801.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://ftp.gwdg.de/pub/languages/perl/CPAN/authors/id/R/RI/RIBASUSHI/SQL-Abstract-1.80.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://ftp.gwdg.de/pub/languages/perl/CPAN/authors/id/R/RS/RSCHUPP/Module-ScanDeps-1.17.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://ftp.gwdg.de/pub/languages/perl/CPAN/authors/id/S/SH/SHLOMIF/Text-Table-1.129.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://ftp.gwdg.de/pub/languages/perl/CPAN/authors/id/S/SZ/SZABGAB/Class-Base-0.05.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://home.comcast.net/~wb2osz/Version%201.2/direwolf-1.2-src.zip’: HTTP response code said error (22)
error: unable to download ‘http://mirror.switch.ch/ftp/mirror/exim/exim/exim4/exim-4.85.tar.bz2’: HTTP response code said error (22)
error: unable to download ‘http://omake.metaprl.org/downloads/omake-0.9.8.6-0.rc1.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://prdownloads.sourceforge.net/eaglemode/eaglemode-0.86.0.tar.bz2’: HTTP response code said error (22)
error: unable to download ‘http://prdownloads.sourceforge.net/indi/libindi_1.0.0.tar.gz’: HTTP response code said error (22)
error: unable to download ‘https://download.electrum.org/Electrum-2.4.tar.gz’: HTTP response code said error (22)
error: unable to download ‘https://download.jetbrains.com/python/pycharm-community-4.5.3.tar.gz’: HTTP response code said error (22)
error: unable to download ‘https://github.com/andmarti1424/scim/archive/v0.1.9.tar.gz’: HTTP response code said error (22)
error: unable to download ‘https://github.com/freenet/fred-official/tarball/eccc9b3198’: HTTP response code said error (22)
error: unable to download ‘https://gitorious.org/beret/beret/archive-tarball/ae029777’: HTTP response code said error (22)
error: unable to download ‘https://python-nbxmpp.gajim.org/downloads/8’: Peer certificate cannot be authenticated with given CA certificates (60)
error: unable to download ‘https://svn.neo-layout.org/linux/console/neo.map?r=2455’: Peer certificate cannot be authenticated with given CA certificates (60)
error: unable to download ‘http://webloria.loria.fr/~quinson/Teaching/PLM/plm-2_5.jar’: HTTP response code said error (22)
error: unable to download ‘http://www.colordiff.org/colordiff-1.0.15.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://www.dicollecte.org/download/fr/hunspell-french-dictionaries-v5.3.zip’: HTTP response code said error (22)
error: unable to download ‘http://www.fon.hum.uva.nl/praat/praat5417_sources.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://www.linux-phc.org/forum/download/file.php?id=167’: HTTP response code said error (22)
error: unable to download ‘http://www.rastersoft.com/descargas/devede-3.23.0.tar.bz2’: HTTP response code said error (22)
error: unable to download ‘http://www.schedmd.com/download/latest/slurm-14.11.5.tar.bz2’: HTTP response code said error (22)
error: unable to download ‘http://www.vips.ecs.soton.ac.uk/supported/current/vips-8.0.2.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://www.w1hkj.com/downloads/fldigi/fldigi-3.22.02.tar.gz’: HTTP response code said error (22)
error: unable to download ‘http://znc.in/releases/znc-1.6.1.tar.gz’: HTTP response code said error (22)

The CA certificate errors are really a nix-prefetch-url bug - it shouldn't complain about certificates if it knows the output hash.

There were also a bunch of files that failed due to invalid characters in the file name, but that's more a bug in copy-tarballs.pl.

[copumpkin]

These stand out as not actually being immutable URLs:

• https://github.com/adobe-fonts/source-code-pro/archive/2.010R-ro/1.030R-it.tar.gz
• https://github.com/namecoin/namecoin/archive/nc0.3.80.tar.gz
• https://github.com/numixproject/numix-icon-theme-circle/archive/d7e8c4cdcf.zip
• https://github.com/ocaml/ocaml/pull/117.patch
• https://github.com/valr/cbatticon/archive/1.4.2.zip
• http://voxelands.com/downloads/voxelands-1506.00-src

[vbgl]

The issue with 117.patch has been fixed in master by a3fa690fa288567662d10d664d89f0e9cd21014d

The one with indilib in 0a745f5dd20f3277c78a7deff875bc6954d2ed59

Should these commits be cherry-picked to the 15.09 branch?

[vcunat]

I resolved those two failures.

[thall]

Reported about colordiff couple of days ago (https://github.com/NixOS/nixpkgs/issues/12656)

[magnetophon]

The ladspa one is fixed here: https://github.com/NixOS/nixpkgs/pull/12640

[4z3]

Looks like for exim only the most current release is kept at the defined location, all other releases get moved to a subdirectory called old. I'm not sure how to fix this besides keeping the package up-to-date.

[vcunat]

@4z3: both/multiple URLs can be specified. They're tried in order.

[joachifm]

Is this still relevant?

[NeQuissimus]

We should try this again before 17.03 to get rid of some of the old garbage we carry...