AlmaLinux / cloud-images

Packer templates and other tools for building AlmaLinux images for various cloud platforms.
MIT License
161 stars 48 forks source link

AlmaLinux-9-GenericCloud-latest.x86_64.qcow2 failing to download #193

Closed rfparedes closed 3 days ago

rfparedes commented 4 days ago

I have automated process that downloads: https://repo.almalinux.org/almalinux/9/cloud/x86_64/images/AlmaLinux-9-GenericCloud-latest.x86_64.qcow2

This is failing to download in entirety. I've tried from multiple locations and servers.

The https://repo.almalinux.org/almalinux/9/cloud/x86_64/images/AlmaLinux-9-GenericCloud-9.5-20241120.x86_64.qcow2 downloads fine.

LKHN commented 3 days ago

@rfparedes ,

Thanks for letting us know. I cannot reproduce the issue and working with the Infrastructure SIG to find out the root of the issue.

codyro commented 3 days ago

This is now resolved. Since the *-latest files are symlinks that point to the latest release, I believe that someone requested the file (symlink) while it wasn't finished syncing, causing it to cache a partial file with the wrong content-length, making everything go haywire. I went ahead and checked the Fastly VCL configuration and saw nothing wrong with it. I verified that the downloads worked fine when bypassing Fasly.

Purging the chained service (that handles the caching) seems to have resolved the issue (and verified the content-length was incorrect).

root@cody-rpm ~ [1]# curl -I https://repo.almalinux.org/almalinux/9/cloud/x86_64/images/AlmaLinux-9-GenericCloud-latest.x86_64.qcow2
HTTP/2 200
cache-control: public, max-age=31536000, immutable
content-type: application/octet-stream
last-modified: Wed, 20 Nov 2024 19:32:08 GMT
etag: "673e3938-1d440000"
accept-ranges: bytes
server: nginx/1.22.1
age: 16
via: 1.1 varnish, 1.1 varnish, 1.1 varnish, 1.1 varnish
date: Sun, 24 Nov 2024 17:56:22 GMT
x-served-by: cache-bur-kbur8200052-BUR, cache-bur-kbur8200055-BUR
x-cache: HIT, MISS, MISS, MISS
x-cache-hits: 1, 0, 0, 0
x-timer: S1732470982.447900,VS0,VE328
content-length: 490995712 <--- WRONG
root@cody-rpm ~# curl -I https://repo.almalinux.org/almalinux/9/cloud/x86_64/images/AlmaLinux-9-GenericCloud-latest.x86_64.qcow2
HTTP/2 200
cache-control: public, max-age=31536000, immutable
content-type: application/octet-stream
last-modified: Mon, 05 Aug 2024 07:54:29 GMT
etag: "66b08535-23450000"
accept-ranges: bytes
server: nginx/1.22.1
age: 443014
via: 1.1 varnish, 1.1 varnish, 1.1 varnish, 1.1 varnish
date: Sun, 24 Nov 2024 17:32:14 GMT
x-served-by: cache-bur-kbur8200112-BUR, cache-bur-kbur8200151-BUR
x-cache: HIT, HIT, MISS, MISS
x-cache-hits: 89, 0, 0, 0
x-timer: S1732469535.885754,VS0,VE17
content-length: 591724544 <--- RIGHT
Linux-9-GenericCloud-latest.x86_64.qcow2
AlmaLinux-9-GenericC 100% [=================================================================================================================>]  468.25M  113.49MB/s
                          [Files: 1  Bytes: 468.25M [105.84MB/s] Redirects: 0  Todo: 0  Errors: 0                                            ]
root@cody-rpm ~# openssl sha256 AlmaLinux-9-GenericCloud-latest.x86_64.qcow2
SHA2-256(AlmaLinux-9-GenericCloud-latest.x86_64.qcow2)= abddf01589d46c841f718cec239392924a03b34c4fe84929af5d543c50e37e37

Thanks for the report! Let me know if you run into any further issues.

rfparedes commented 3 days ago

@codyro Thanks Cody. I can confirm also, all is good now.