mopidy-bandcamp / python3-urllib3: connect to bandcamp.com gets error 403

woffs commented 1 month ago

Describe the bug

mopidy-bandcamp ceased to work as shown in https://github.com/impliedchaos/mopidy-bandcamp/issues/25 It turned out that a change in urllib3 as shown in https://github.com/urllib3/urllib3/issues/3439 triggered this. The error is reproducible in NixOS, but not in Debian.

Steps To Reproduce

$ nix-shell -p 'python3.withPackages (p: with p; [ p.urllib3 ])' --run 'python -c "import urllib3; print(urllib3.request(\"GET\",\"https://bandcamp.com/api/fan/2/collection_summary\").status)"'
403

copied from https://github.com/urllib3/urllib3/issues/3439 :

Expected Behavior

~/git/urllib3 (1.26.19)$ rm -r /tmp/u ; nix run nixpkgs#python3Packages.pip -- install --prefix /tmp/u .
~/git/urllib3 (1.26.19)$ PYTHONPATH=/tmp/u/lib/python3.11/site-packages/ nix run nixpkgs#python3 -- -c 'import urllib3;print(urllib3.PoolManager().request("GET","https://bandcamp.com/api/fan/2/collection_summary").status)'
200

Actual Behavior

~/git/urllib3 (main)$ rm -r /tmp/u ; nix run nixpkgs#python3Packages.pip -- install --prefix /tmp/u .
~/git/urllib3 (main)$ PYTHONPATH=/tmp/u/lib/python3.11/site-packages/ nix run nixpkgs#python3 -- -c 'import urllib3;print(urllib3.request("GET","https://bandcamp.com/api/fan/2/collection_summary").status)'
403

Additional Context

I do not know which detail of TLS connection settings triggers this funny behaviour at bandcamp.com. All connection attempts using curl or openssl s_connect work seamlessly and result in a 200 response. The 403 answer is not reproducible outside python-in-nixos yet.

Metadata

system: "x86_64-linux"
host os: Linux 6.6.43, NixOS, 24.05 (Uakari), 24.05.20240803.704b2f6
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.18.5
channels(root): ""
channels(woffs): ""
nixpkgs: /nix/store/sp1k11m59c8lp0d1y14rgw325zan6p81-source

Add a :+1: reaction to issues you find important.

woffs commented 3 weeks ago

so how to morph https://github.com/urllib3/urllib3/issues/3439#issuecomment-2306400349 into a proper fix/workaround/patch here?

woffs commented 1 week ago

using https://gist.github.com/woffs/083f57f6cd0f4bfe49a6e97e1eb2ed60 and extensionPackages = with pkgs; [ (mopidy-bandcamp.overrideAttrs (o: { patches = ./bandcamp.patch; })) ];

jpotier commented 1 week ago

Many thanks @woffs I can finally listen to bandcamp again :heart:

I went with the overlay route, which works closely enough the same (I'm just used to it):

{...}:
{
  nixpkgs.overlays = [
    # 2024-09-08
    # https://github.com/NixOS/nixpkgs/issues/332223
    (_: prev: {
      mopidy-bandcamp = prev.mopidy-bandcamp.overrideAttrs (
        _: {patches = ../../data/bandcamp.patch;}
      );
    })
  ];

NixOS / nixpkgs