pacman 6: breaks getting .sig files from GitHub releases

[jeremyd2019]

Background: I have been using a GitHub release to host my 32-bit msys2 builds for almost a year. More recently, I used them to host bootstrapping stages for clang, and am still using one to host clangarm64 which I have started signing.

Testing pacman 6.0 which was recently merged (but not yet out of staging) I started getting errors of the form:

error: could not open file /var/cache/pacman/pkg/7f8ba900-b620-11eb-847e-bea3b25d361c?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210607%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210607T020754Z&X-Amz-Expires=300&X-Amz-Signature=1526d03d32326d8f7bdf9aa3077cff690d28a5d99b9e62ed70e68dd700aee445&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=284102925&response-content-disposition=attachment%3B%20filename%3Dpython-setuptools-47.1.1-2-any.pkg.tar.zst&response-content-type=application%2Foctet-stream.sig.part: File name too long

I have done more testing, and it seems that this also breaks the now-signed clangarm64 repo if I switch it from Optional for the test:

error: could not open file /var/lib/pacman/sync/7db34980-c458-11eb-94bf-fc73fd3e09fe?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210607%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210607T044405Z&X-Amz-Expires=300&X-Amz-Signature=efe2f90a8e4bd7f79c00b08f32f6c389d1c3e4a3bb3eecb8ead7f1857683a316&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=308011265&response-content-disposition=attachment%3B%20filename%3Dclangarm64.db&response-content-type=application%2Foctet-stream.sig.part: File name too long
error: clangarm64: missing required signature
error: failed to synchronize all databases (unexpected error)

I am guessing that pacman 6 appends .sig to the URL after redirects are processed now rather than before, which breaks with GitHub's crazy long S3 URLs.

[lazka]

I guess somehow related to https://git.archlinux.org/pacman.git/commit/lib/libalpm/dload.c?id=a8bdc2e10ad6c942f3f1dbbbb77c126578a41bd3 ?

[jeremyd2019]

That's probably it. But it looks from that long name that it is trying to set content-disposition, which should have had precedence over the "name" of all the query string parameters.

[jeremyd2019]

Nope, I think it's actually https://github.com/msys2/msys2-pacman/commit/f078c2d3bcb72bafda0dce5fe2c9418ca462bb1a:

To build *.sig file url we've been using a simple string concatenation: $requested_url + ".sig". Unfortunately there are cases when it does not work. For example an archlinux.org "Download From Mirror" link looks like this https://www.archlinux.org/packages/core/x86_64/bash/download/ and it gets redirected to some mirror. But if we append ".sig" to the end of the link url and try to download it then archlinux.org returns 404 error.

To overcome this issue we need to follow redirects for the main payload first, find the final url and only then append '.sig' suffix.

[lazka]

https://bugs.archlinux.org/task/71148

[jeremyd2019]

Nice, at least it's not just me for a change :wink:

[lazka]

There is a proposed upstream patch now: https://lists.archlinux.org/pipermail/pacman-dev/2021-June/025222.html

[jeremyd2019]

The most recent version of this patch seems to be https://lists.archlinux.org/pipermail/pacman-dev/2021-June/025252.html

[lazka]

I think sourceforge is also affected by this after https://github.com/msys2/MSYS2-packages/pull/2571

[lempamo]

yep. ran into this when trying to install gtk4. specifically happened with the following dependencies:

mingw-w64-x86_64-chromaprint-1.5.0-1
mingw-w64-x86_64-gst-plugins-bad-1.18.4-7
mingw-w64-x86_64-libbs2b-3.1.0-1
mingw-w64-x86_64-libdca-0.0.7-1
mingw-w64-x86_64-libdvdnav-6.1.0-2
mingw-w64-x86_64-libmpeg2-git-r1108.946bf4b-1
mingw-w64-x86_64-libnice-0.1.18-2
mingw-w64-x86_64-soundtouch-2.2-1
mingw-w64-x86_64-vo-amrwbenc-0.1.3-1
mingw-w64-x86_64-webrtc-audio-processing-0.3.1-1

[jeremyd2019]

I'm surprised that this hasn't gotten any more attention upstream by now.

[jeremyd2019]

It looks like some movement is happening upstream now. https://lists.archlinux.org/pipermail/pacman-dev/2021-September/025316.html

[lazka]

fixed in 6.0.1

[jeremyd2019]

Cool, I kind of thought a 6.0.1 was imminent because patches had broken loose upstream.