tor-browser-bundle-bin 8.5.4 no longer available from mirrors

[hkjn]

Description

(Similar to #64361, which seems outdated?)

The 8.5.4 version of tor-browser-bundle-bin package does not build at current master (https://github.com/NixOS/nixpkgs/commit/2e47cb88ea081af47df986f17df341ac0265a8a4).

To Reproduce Steps to reproduce the behavior:

1. git clone https://github.com/NixOS/nixpkgs .
2. nix-rebuild switch -I $(pwd) --upgrade
3. nix-env -iA nixpkgs.tor-browser-bundle-bin

Attempting to install the package fails:

$ nix-env -iA nixpkgs.tor-browser-bundle-bin

replacing old 'tor-browser-bundle-bin-9.0a6'
installing 'tor-browser-bundle-bin-8.5.4'
these derivations will be built:

/nix/store/bs1v8mzd0hkix495ph0lgxslpyjhpgz6-tor-browser-linux64-8.5.4_en-US.tar.xz.drv
[...]
building '/nix/store/bs1v8mzd0hkix495ph0lgxslpyjhpgz6-tor-browser-linux64-8.5.4_en-US.tar.xz.drv'...

trying https://github.com/TheTorProject/gettorbrowser/releases/download/v8.5.4/tor-browser-linux64-8.5.4_en-US.tar.xz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404 Not Found

trying https://dist.torproject.org/torbrowser/8.5.4/tor-browser-linux64-8.5.4_en-US.tar.xz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404 Not Found
error: cannot download tor-browser-linux64-8.5.4_en-US.tar.xz from any mirror
builder for 
'/nix/store/bs1v8mzd0hkix495ph0lgxslpyjhpgz6-tor-browser-linux64-8.5.4_en-US.tar.xz.drv' 
failed with exit code 1
cannot build derivation '/nix/store/xwga10zf487k81qws69ry2vj0ksgp42v-tor-browser-bundle-bin-8.5.4.drv': 1 
dependencies couldn't be built
error: build of '/nix/store/xwga10zf487k81qws69ry2vj0ksgp42v-tor-browser-bundle-bin-8.5.4.drv' failed

Expected behavior

Package installs successfully.

Additional context

Viewing https://dist.torproject.org/torbrowser/, versions 8.5.5, 8.5.6 and 9.0a6 are the only one accessible from that mirror. I'm not sure why the earlier released versions no longer are published.

Metadata

$ nix run nixpkgs.nix-info -c nix-info -m
 - system: `"x86_64-linux"`
 - host os: `Linux 4.19.67, NixOS, 19.03.git.0c60407 (Koi)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.2.2`
 - channels(root): `"nixos-19.03.173486.cf018a7c558"`
 - channels(user): `""`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute: tor-browser-bundle-bin
# a list of nixos modules affected by the problem
module:

[gloaming]

Viewing https://dist.torproject.org/torbrowser/, versions 8.5.5, 8.5.6 and 9.0a6 are the only one accessible from that mirror. I'm not sure why the earlier released versions no longer are published.

I expect that the Tor project does not distribute old versions of their software due to its security-critical nature. Also, because it is networking software, there are probably compatibility concerns.

As for the GitHub repo, it's a mess and has not updated properly in a year, so it doesn't have releases later than 8.0.2.

[hkjn]

I expect that the Tor project does not distribute old versions of their software due to its security-critical nature. Also, because it is networking software, there are probably compatibility concerns.

That makes sense, however feels like a pain to have packages break when no changes are made to anything in the .nix world.. I'd expect that with the same expressions, ideally identical build outputs would result.

As for the GitHub repo, it's a mess and has not updated properly in a year, so it doesn't have releases later than 8.0.2.

You mean https://github.com/TheTorProject/gettorbrowser/releases/? Yeah, I noticed that and was confused until I noticed the other mirror (https://dist.torproject.org/torbrowser/)..

Do you think it would make sense to drop the github.com repo as a mirror entirely?

[gloaming]

That makes sense, however feels like a pain to have packages break when no changes are made to anything in the .nix world.. I'd expect that with the same expressions, ideally identical build outputs would result.

True, although ultimately Nix can only provide identical outputs when the inputs are also identical, which is why we check the hashes. Normally, we take pains to ensure the inputs will be available, but in this specific case, I think it may be better to let the package break. We should ask someone at the Tor project for an opinion on that.

Do you think it would make sense to drop the github.com repo as a mirror entirely?

Yeah, I think so.

[hkjn]

Do you think it would make sense to drop the github.com repo as a mirror entirely?

Yeah, I think so.

Okay, I've sent a PR to drop the github.com mirror:

• https://github.com/NixOS/nixpkgs/pull/68632

[joachifm]

The problem of disappearing source could be alleviated by re-enabling Hydra builds for tor-browser-bundle-bin; then source tarballs should be mirrored to tarballs.nixos.org.

[Kiwi]

I expect that the Tor project does not distribute old versions of their software due to its security-critical nature. Also, because it is networking software, there are probably compatibility concerns.

Here's an archive of every release since forever. Maybe we could/should use?

As for the GitHub repo, it's a mess and has not updated properly in a year, so it doesn't have releases later than 8.0.2.

Well, their CI puts a version here (currently ~15 day old 8.5.4) probably not helpful... that archive might help, though.

[joachifm]

@Kiwi sounds good to me

[gloaming]

I'm still concerned about the security impact. Would it be possible to at least warn the user if the release has to be fetched from the archive?

[joachifm]

@gloaming I think the best we can do is to actively maintain the package, like we have to do for other packages whose source does not disappear upon new releases (e.g., tor itself, firefox, linux). For known vulnerable versions, we can add insecure markings.

[Kiwi]

Not that my opinion counts for much but I was pondering; what if we added a new package that used the archive in addition to what we have now and also be vigilant with updates. The failing builds of the existing one would be canaries that the archive package needs updating, in case of lack of vigilance. That way users can always have a tor-browser (even if slightly older), which is still better than no tor-browser. $0.02

[gloaming]

@joachifm Yeah, I guess ultimately there's not much we can do to protect users who don't update. The analogy to linux is pretty convincing.

@Kiwi That's an interesting idea, but implementing it as a package doesn't seem quite the right way. I guess a warning from CI would be more appropriate, but I don't think we have any existing process for that.

[timoleistner]

I'm still having the same problem mentioned by @hkjn. Used same steps to reproduce, same error occurs.

nix-env -iA nixos.tor-browser-bundle-bin

installing 'tor-browser-bundle-bin-10.0.8'
these derivations will be built:
  /nix/store/1xzmsgvh39gcahwnm00xcdg5qfzhmyjj-torbrowser.desktop.drv
  /nix/store/zf77222kiyf0vzjaqp62ri89dsrbr4qh-tor-browser-linux64-10.0.8_en-US.tar.xz.drv
  /nix/store/8gq3ys7bpqq2indgrfyshxr1id8rz18l-tor-browser-bundle-bin-10.0.8.drv
building '/nix/store/1xzmsgvh39gcahwnm00xcdg5qfzhmyjj-torbrowser.desktop.drv'...
Running desktop-file validation
/nix/store/b61ybf18ny9wyrzg3100wcjh49r6p2fk-torbrowser.desktop/share/applications/torbrowser.desktop: hint: value item "Security" in key "Categories" in group "Desktop Entry" can be extended with another category among the following categories: Settings, or System
building '/nix/store/zf77222kiyf0vzjaqp62ri89dsrbr4qh-tor-browser-linux64-10.0.8_en-US.tar.xz.drv'...

trying https://dist.torproject.org/torbrowser/10.0.8/tor-browser-linux64-10.0.8_en-US.tar.xz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404 Not Found
error: cannot download tor-browser-linux64-10.0.8_en-US.tar.xz from any mirror
builder for '/nix/store/zf77222kiyf0vzjaqp62ri89dsrbr4qh-tor-browser-linux64-10.0.8_en-US.tar.xz.drv' failed with exit code 1
cannot build derivation '/nix/store/8gq3ys7bpqq2indgrfyshxr1id8rz18l-tor-browser-bundle-bin-10.0.8.drv': 1 dependencies couldn't be built
error: build of '/nix/store/8gq3ys7bpqq2indgrfyshxr1id8rz18l-tor-browser-bundle-bin-10.0.8.drv' failed

More info:

nix run nixpkgs.nix-info -c nix-info -m

 - system: `"x86_64-linux"`
 - host os: `Linux 5.4.89, NixOS, 20.09.2996.002c001ab6f (Nightingale)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.10`
 - channels(root): `"nixos-20.09.2996.002c001ab6f"`
 - channels(timo): `"home-manager-20.09"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[msm-code]

@timoleistner and for anyone else stumbling upon this issue (like I did).

Why this happens: download URL for tor browser is hardcoded in nixpkgs. In unstable it's updated pretty frequently, but can get out of date for stable channels. And tor browser people aggresively remove their outdated releases (for good reasons).

To fix this, I've created (my first) overlay tor-browser-fixup.nix:

self: super:
{
  tor-browser-bundle-bin = super.tor-browser-bundle-bin.overrideAttrs (old: {
    src = super.fetchurl {
        urls = [
            "https://dist.torproject.org/torbrowser/11.0.4/tor-browser-linux64-11.0.4_en-US.tar.xz"
            "https://tor.eff.org/dist/torbrowser/11.0.4/tor-browser-linux64-11.0.4_en-US.tar.xz"
        ];
        sha256 = "0pz1v5ig031wgnq3191ja08a4brdrbzziqnkpcrlra1wcdnzv985";
    };
  });
}

I just copied the most recent URLs and hash from https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/networking/browsers/tor-browser-bundle-bin/default.nix (but you can use any version you like).

Use this as an overlay (see also: https://nixos.wiki/wiki/Overlays). In my case this means adding this to my main flake.nix:

nixpkgs.overlays = [
  (import ./tor-browser-fixup.nix)
];

And you should be able to use tor-browser.