Getting 403 error randomly when installing temurin packages

[angaaruriakhil]

We are conducting ubuntu 20.04 image builds using code in the runner-images repository.

When installing temurin JDK packages, we now get 403 errors randomly on differing packages.

We conducted a build yesterday which failed on installing temurin 21 with the following error:

azure-arm: E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-21/temurin-21-jdk_21.0.0.0.0+35_amd64.deb 403 Forbidden [IP: 146.75.107.42 443]

We conducted a build today which failed on temurin 11 with the following error:

azure-arm.build_vhd: E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-11/temurin-11-jdk_11.0.20.1.0%2b1_amd64.deb Error reading from server. Remote end closed connection [IP: 146.75.107.42 443]

We have also had some successful builds where all the packages install.

To Reproduce

Run code in java-tools.sh to install temurin packages.

Expected behavior Every package is able to be installed reliably

[karianna]

@angaaruriakhil Can you try updating you ca-certificates package and try again?

[KrustyDaClown]

Got the same issue here - even after sudo apt-get install ca-certificates; sudo update-ca-certificates

It worked after an sudo apt update --fix-missing

[karianna]

@gdams over to you - I can only imagine a CDN issue?

[jlgoolsbee]

We are intermittently seeing this issue as well, specifically via yum but I was able to reproduce (albeit with somewhat different error messaging) with curl as well (output below).

Example yum install failure:

$ sudo yum install -y temurin-11-jdk
Loaded plugins: extras_suggestions, langpacks, priorities, update-motd

# [ dependency resolution omitted for brevity's sake ]

Total size: 160 M
Total download size: 159 M
Installed size: 314 M
Downloading packages:
No Presto metadata available for Adoptium
temurin-11-jdk-11.0.20.1.0.1-1 FAILED                                          
https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-11-jdk-11.0.20.1.0.1-1.x86_64.rpm: [Errno 14] HTTPS Error 403 - Forbidden                                                                                                 ]  0.0 B/s |    0 B  --:--:-- ETA 
Trying other mirror.

Error downloading packages:
  temurin-11-jdk-11.0.20.1.0.1-1.x86_64: [Errno 256] No more mirrors to try.

I encountered this failure several times over starting at 14:55 UTC today, but in my case only for the temurin-11-jdk package; I was able to successfully download and install temurin-11-jre as well as temurin-17-jdk, but temurin-11-jdk continued to fail. As had been suggested on this issue and issue #554, I tried both updating the ca-certificates package and running update-ca-trust – neither of which updated anything – but installation of temurin-11-jdk continued to fail. At this point I tried to curl the failing URL, and I did not receive a 403, but I also wasn't able to download the RPM:

$ curl -vLO https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-11-jdk-11.0.20.1.0.1-1.x86_64.rpm
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 146.75.39.42:443...
* Connected to packages.adoptium.net (146.75.39.42) port 443
* ALPN: curl offers h2,http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
*  CAfile: /etc/pki/tls/certs/ca-bundle.crt
*  CApath: none
{ [5 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [100 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [3978 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [333 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=packages.adoptium.net
*  start date: Oct  2 11:22:18 2023 GMT
*  expire date: Dec 31 11:22:17 2023 GMT
*  subjectAltName: host "packages.adoptium.net" matched cert's "packages.adoptium.net"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
} [5 bytes data]
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-11-jdk-11.0.20.1.0.1-1.x86_64.rpm
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: packages.adoptium.net]
* [HTTP/2] [1] [:path: /artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-11-jdk-11.0.20.1.0.1-1.x86_64.rpm]
* [HTTP/2] [1] [user-agent: curl/8.3.0]
* [HTTP/2] [1] [accept: */*]
} [5 bytes data]
> GET /artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-11-jdk-11.0.20.1.0.1-1.x86_64.rpm HTTP/2
> Host: packages.adoptium.net
> User-Agent: curl/8.3.0
> Accept: */*
> 
{ [5 bytes data]
< HTTP/2 200 
< x-amz-id-2: H5ntxCxOdyFhTEZa+JrnjbA2V9VdbpXoFdCeekzjgSGgWSBi/YUbTChNj90JUOuqUxyWHrANR+A=
< x-amz-request-id: QJZ3FQGCMJW8Z84N
< x-amz-replication-status: FAILED
< last-modified: Thu, 31 Aug 2023 12:51:43 GMT
< etag: "71b38a94dbeff6f692cf13a3b7038f35"
< x-amz-server-side-encryption: AES256
< x-amz-version-id: umzWdTNJmMDdvnR0unSw_fRTy4j1NfZd
< content-disposition: attachment;filename="temurin-11-jdk-11.0.20.1.0.1-1.x86_64.rpm"
< content-type: application/x-rpm
< server: AmazonS3
< cache-control: public, max-age=15780096.000
< fastly-restarts: 1
< accept-ranges: bytes
< date: Tue, 24 Oct 2023 15:13:19 GMT
< via: 1.1 varnish
< age: 3022
< x-served-by: cache-iad-kcgs7200148-IAD
< x-cache: HIT
< x-cache-hits: 1
< x-timer: S1698160399.397782,VS0,VE2
< strict-transport-security: max-age=31557600
< content-length: 166352397
< 
{ [5 bytes data]
* HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)
 60  158M   60 96.0M    0     0   166M      0 --:--:-- --:--:-- --:--:--  166M
* Connection #0 to host packages.adoptium.net left intact
curl: (92) HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)

This failed command actually did download 96M – other attempts downloaded different amounts – but it always failed. The error message implying that the stream wasn't closed cleanly leads me to wonder if the cached asset is corrupt, or perhaps isn't fully-distributed/cached by the CDN? I'm not sure if/how yum is interpreting this as a 403, or if this error is somehow unique/different... meanwhile, downloading the RPM for temurin-17-jdk via curl worked just fine:

$ curl -vLO https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.8.1.0.1-1.x86_64.rpm
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 146.75.35.42:443...
* Connected to packages.adoptium.net (146.75.35.42) port 443
* ALPN: curl offers h2,http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
*  CAfile: /etc/pki/tls/certs/ca-bundle.crt
*  CApath: none
{ [5 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [100 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [3978 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [333 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=packages.adoptium.net
*  start date: Oct  2 11:22:18 2023 GMT
*  expire date: Dec 31 11:22:17 2023 GMT
*  subjectAltName: host "packages.adoptium.net" matched cert's "packages.adoptium.net"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
} [5 bytes data]
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.8.1.0.1-1.x86_64.rpm
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: packages.adoptium.net]
* [HTTP/2] [1] [:path: /artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.8.1.0.1-1.x86_64.rpm]
* [HTTP/2] [1] [user-agent: curl/8.3.0]
* [HTTP/2] [1] [accept: */*]
} [5 bytes data]
> GET /artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.8.1.0.1-1.x86_64.rpm HTTP/2
> Host: packages.adoptium.net
> User-Agent: curl/8.3.0
> Accept: */*
> 
{ [5 bytes data]
< HTTP/2 200 
< x-amz-id-2: xUHOk3FxG3CALkYV/8zV6BjfkJl7o9PVSMO6jAXu1BBEKzEi8qRBbhPN+6SrCDSncmHJE9JYhqc=
< x-amz-request-id: 9FDX85EN3B8YFEC4
< x-amz-replication-status: FAILED
< last-modified: Thu, 31 Aug 2023 14:32:03 GMT
< etag: "a8ed74787d123dc1a9664b59a4c14ca2"
< x-amz-server-side-encryption: AES256
< x-amz-version-id: Z1zAzlEUB_ubAmeOULSp6EX2L.7XXn9A
< content-disposition: attachment;filename="temurin-17-jdk-17.0.8.1.0.1-1.x86_64.rpm"
< content-type: application/x-rpm
< server: AmazonS3
< cache-control: public, max-age=15780096.000
< fastly-restarts: 1
< accept-ranges: bytes
< date: Tue, 24 Oct 2023 16:12:30 GMT
< via: 1.1 varnish
< age: 0
< x-served-by: cache-iad-kjyo7100121-IAD
< x-cache: MISS
< x-cache-hits: 0
< x-timer: S1698163950.013750,VS0,VE913
< strict-transport-security: max-age=31557600
< content-length: 162689473
< 
{ [5 bytes data]
100  155M  100  155M    0     0  9496k      0  0:00:16  0:00:16 --:--:-- 10.2M
* Connection #0 to host packages.adoptium.net left intact

Diffing the two downloads, I noted this first download of _temurin-17-jdk-17.0.8.1.0.1-1.x8664.rpm showed a cache MISS; subsequent requests for the same URL were also successful and showed a cache HIT (as did all my requests for _temurin-11-jdk-11.0.20.1.0.1-1.x8664.rpm), but otherwise they appear functionally similar (no appreciable difference that might point to a reason for the "stream 1 was not closed cleanly" errors that I can see, anyways).

I lost track of the exact time while I was debugging all this, but at some point between 20-30 minutes later, a stray sudo yum install -y temurin-11-jdk completed successfully, and as I can't point to anything I did which resolved this issue on my end, I'm left to assume this issue must be present somewhere on the Artifactory/S3/Fastly-side of things.

[angaaruriakhil]

Thanks for the responses folks,

I have not seen this issue this week on our builds, we've had 4 successful builds with temurin 8, 11, 17, and 21 installing on each build without issue. Haven't changed anything in our code since my last post. I'm in agreement with the above that it's possibly artifactory/CDN related...

[jlgoolsbee]

I don't agree that this issue should be closed – I experienced this issue just yesterday, as documented above – and even if we accept the premise that this is a transient issue, it is still a persistent issue that deserves further investigation.

[KrustyDaClown]

Got the issue again. This time a variable ${VERSION} in the download URL target seems not to be resolved:

olger@phobos:~$ sudo apt upgrade
Paketlisten werden gelesen… Fertig
Abhängigkeitsbaum wird aufgebaut… Fertig
Statusinformationen werden eingelesen… Fertig
Paketaktualisierung (Upgrade) wird berechnet… Fertig
Die folgenden Pakete werden aktualisiert (Upgrade):
  temurin-17-jdk temurin-21-jdk
2 aktualisiert, 0 neu installiert, 0 zu entfernen und 0 nicht aktualisiert.
Es müssen 343 MB an Archiven heruntergeladen werden.
Nach dieser Operation werden 197 kB Plattenplatz zusätzlich benutzt.
Möchten Sie fortfahren? [J/n] 
Fehl:1 https://packages.adoptium.net/artifactory/deb bookworm/main amd64 temurin-17-jdk amd64 17.0.9.0+9
  400  Bad Request [IP: 146.75.123.42 443]
Fehl:2 https://packages.adoptium.net/artifactory/deb bookworm/main amd64 temurin-21-jdk amd64 21.0.1.0.0+12
  400  Bad Request [IP: 146.75.123.42 443]
E: Fehlschlag beim Holen von https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-${VERSION}/temurin-17-jdk_17.0.9.0%2b9_amd64.deb 400  Bad Request [IP: 146.75.123.42 443]
E: Fehlschlag beim Holen von https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-${VERSION}/temurin-21-jdk_21.0.1.0.0%2b12_amd64.deb 400  Bad Request [IP: 146.75.123.42 443]
E: Einige Archive konnten nicht heruntergeladen werden; vielleicht »apt-get update« ausführen oder mit »--fix-missing« probieren?

Is says "failed to get" https//packages.adoptium.net/artifactory/deb/pool/main/t/temurin-${VERSION}/temurin-17-jdk_17.0.9.0%2b9_amd64.deb https//packages.adoptium.net/artifactory/deb/pool/main/t/temurin-${VERSION}/temurin-21-jdk_21.0.1.0.0%2b12_amd64.deb and I guess the string "${VERSION}" is not intended to be part of the URL...

[geenweb]

I'm having the same issue on Linux Mint 20 (Ubuntu 20.04)

The following packages will be upgraded:
  temurin-17-jdk temurin-17-jre
2 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 195 MB of archives.
After this operation, 2,048 B disk space will be freed.
Err:1 https://packages.adoptium.net/artifactory/deb focal/main amd64 temurin-17-jdk amd64 17.0.9.0+9
  400  Bad Request [IP: 151.101.131.42 443]
Err:2 https://packages.adoptium.net/artifactory/deb focal/main amd64 temurin-17-jre amd64 17.0.9.0+9
  400  Bad Request [IP: 151.101.131.42 443]
E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-${VERSION}/temurin-17-jdk_17.0.9.0+9_amd64.deb  400  Bad Request [IP: 151.101.131.42 443]
E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-${VERSION}/temurin-17-jre_17.0.9.0+9_amd64.deb  400  Bad Request [IP: 151.101.131.42 443]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

I updated certificates and tried --fix-missing. No joy.

[dustingrady]

Same issue here on Ubuntu 22.04 after updating ca-certs and trying --fix-missing

Err:1 https://packages.adoptium.net/artifactory/deb jammy/main amd64 temurin-17-jdk amd64 17.0.9.0+9
  400  Bad Request [IP: 151.101.3.42 443]
E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-${VERSION}/temurin-17-jdk_17.0.9.0%2b9_amd64.deb  400  Bad Request [IP: 151.101.3.42 443]

[karianna]

Hi folks, we're aware of the issue and are investigating with our CDN and package repo providers.

[gdams]

To anyone making an update to a URL that contains -${VERSION} this was an upload error which we've subsequently removed. It appears that artifactory didn't recalculate the index so I've triggered this manually which should fix most user issues. Please can people that were hitting this URL try updating their packages again?

The second bug is around the different behaviour that artifactory serves users if they are in a browser or on a command line. If you make a request in the browser it creates a 301 redirect to a /ui route so that you can explore the repo in artifactory whereas in the command line, it should always serve the binary. I’ve managed to create some code that synthetically handles these 301’s for browser users which should mean that Fastly stops caching the wrong response.

[geenweb]

Works now. Thank you.

[repudi8or]

this is not solved...

wget https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm --2023-10-31 12:38:49-- https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm Resolving packages.adoptium.net (packages.adoptium.net)... 151.101.23.42 Connecting to packages.adoptium.net (packages.adoptium.net)|151.101.23.42|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 162838233 (155M) [application/x-rpm] Saving to: ‘temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm.2’

61% [====================================================================> ] 100,663,296 312MB/s in 0.3s

2023-10-31 12:38:50 (312 MB/s) - Connection closed at byte 100663296. Retrying.

--2023-10-31 12:38:51-- (try: 2) https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm Connecting to packages.adoptium.net (packages.adoptium.net)|151.101.23.42|:443... connected. HTTP request sent, awaiting response... 403 Forbidden 2023-10-31 12:38:51 ERROR 403: Forbidden.

wget https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm --2023-10-31 12:41:40-- https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm Resolving packages.adoptium.net (packages.adoptium.net)... 151.101.23.42 Connecting to packages.adoptium.net (packages.adoptium.net)|151.101.23.42|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 162838233 (155M) [application/x-rpm] Saving to: ‘temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm.3’

61% [====================================================================> ] 100,663,296 294MB/s in 0.3s

2023-10-31 12:41:40 (294 MB/s) - Connection closed at byte 100663296. Retrying.

--2023-10-31 12:41:41-- (try: 2) https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm Connecting to packages.adoptium.net (packages.adoptium.net)|151.101.23.42|:443... connected. HTTP request sent, awaiting response... 403 Forbidden 2023-10-31 12:41:41 ERROR 403: Forbidden. wget --no-cache https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm --2023-10-31 12:43:18-- https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm Resolving packages.adoptium.net (packages.adoptium.net)... 146.75.43.42 Connecting to packages.adoptium.net (packages.adoptium.net)|146.75.43.42|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 162838233 (155M) [application/x-rpm] Saving to: ‘temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm.4’

[angaaruriakhil]

Hi, we have been experiencing this issue again today.

[gdams]

I'm talking with Fastly again today, I think that Fastly is caching incomplete requests and then continuing to serve them rather than routing users directly to Artificatory. I'll provide an update here once I have one.

[xsnrg]

Still having this issue. Failed to fetch temurin-21-jdk_21.0.1.0.0%2b12_amd64.deb.

Error reading from server. Remote end closed connection

[gdams]

Still having this issue. Failed to fetch temurin-21-jdk_21.0.1.0.0%2b12_amd64.deb.

Error reading from server. Remote end closed connection

@xsnrg can you share the full URL that you are trying to request?

[xsnrg]

sure thing. It seems to be working again this morning.

E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-21/temurin-21-jdk_21.0.1.0.0%2b12_amd64.deb  Error reading from server. Remote end closed connection [IP: 146.75.39.42 443]

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

[ryantse]

@gdams: Seems like this issue is happening somewhat frequently for me as well:

22:15:08  ==> x86_64-ami-builder: https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-11-jdk-11.0.20.1.0.1-1.x86_64.rpm: [Errno 14] curl#92 - "HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)"
22:15:08  ==> x86_64-ami-builder: Trying other mirror.
22:15:08  ==> x86_64-ami-builder:
22:15:08  ==> x86_64-ami-builder:
22:15:08  ==> x86_64-ami-builder: Error downloading packages:
22:15:08  ==> x86_64-ami-builder:   temurin-11-jdk-11.0.20.1.0.1-1.x86_64: [Errno 256] No more mirrors to try.
22:15:08  ==> x86_64-ami-builder:

[inodes]

This is inconsistent and still requires attention.

Most of the time HTTPS Error 403 - Forbidden:

temurin-11-jdk-11.0.21.0.0.9-1 FAILED
https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-11-jdk-11.0.21.0.0.9-1.x86_64.rpm: [Errno 14] HTTPS Error 403 - Forbidden                      ]  0.0 B/s |    0 B  --:--:-- ETA
Trying other mirror.

In the very rare case the binary is able to download it often fails with HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)

temurin-11-jdk-11.0.21.0.0.9-1 FAILED                                                       30% [=========================                                                         ]  0.0 B/s |  50 MB  --:--:-- ETA
https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-11-jdk-11.0.21.0.0.9-1.x86_64.rpm: [Errno 14] curl#92 - "HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)"-:-- ETA
Trying other mirror.

When persistent, there will a rare occasion when it works (which some people posting above when it "appears to be working again".... are likely getting the lucky one off.

Total download size: 161 M
Installed size: 315 M
Downloading packages:
Public key for temurin-11-jdk-11.0.21.0.0.9-1.x86_64.rpm is not installed
--------------------------------------------------------------------------------
Total                                              187 MB/s | 161 MB  00:00
Retrieving key from https://packages.adoptium.net/artifactory/api/gpg/key/public
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libXau-1.0.8-2.1.amzn2.0.2.x86_64                           1/10
  Installing : libxcb-1.12-1.amzn2.0.2.x86_64                              2/10
  Installing : alsa-lib-1.1.4.1-2.amzn2.x86_64                             3/10
  Installing : libX11-common-1.6.7-3.amzn2.0.4.noarch                      4/10
  Installing : libX11-1.6.7-3.amzn2.0.4.x86_64                             5/10
  Installing : libXext-1.3.3-3.amzn2.0.2.x86_64                            6/10
  Installing : libXi-1.7.9-1.amzn2.0.2.x86_64                              7/10
  Installing : libXtst-1.2.3-1.amzn2.0.2.x86_64                            8/10
  Installing : libXrender-0.9.10-1.amzn2.0.2.x86_64                        9/10
  Installing : temurin-11-jdk-11.0.21.0.0.9-1.x86_64                      10/10
  Verifying  : libXext-1.3.3-3.amzn2.0.2.x86_64                            1/10
  Verifying  : libXrender-0.9.10-1.amzn2.0.2.x86_64                        2/10
  Verifying  : libX11-common-1.6.7-3.amzn2.0.4.noarch                      3/10
  Verifying  : libX11-1.6.7-3.amzn2.0.4.x86_64                             4/10
  Verifying  : libXtst-1.2.3-1.amzn2.0.2.x86_64                            5/10
  Verifying  : alsa-lib-1.1.4.1-2.amzn2.x86_64                             6/10
  Verifying  : libXau-1.0.8-2.1.amzn2.0.2.x86_64                           7/10
  Verifying  : temurin-11-jdk-11.0.21.0.0.9-1.x86_64                       8/10
  Verifying  : libXi-1.7.9-1.amzn2.0.2.x86_64                              9/10
  Verifying  : libxcb-1.12-1.amzn2.0.2.x86_64                             10/10

Installed:
  temurin-11-jdk.x86_64 0:11.0.21.0.0.9-1

[nchammas]

I am seeing this as well today when installing Adoptium JDK 11 via Flintrock, which installs JDK on instances launched on EC2:

https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-11-jdk-11.0.21.0.0.9-1.x86_64.rpm: [Errno 14] HTTPS Error 403 - Forbidden
Trying other mirror.

Error downloading packages:
  temurin-11-jdk-11.0.21.0.0.9-1.x86_64: [Errno 256] No more mirrors to try.

Is there some more information I can share to help identify the root cause here?

[mdaloia]

I was getting 403 Forbidden yesterday, some minutes later it worked ok without doing anything different. This is a very unreliable and flaky networking issue.

I got the chance to get some information in case it is useful to find the root cause by using Ubuntu APT debugging options (-oDebug::Acquire::https=1, for example apt-get install -oDebug::Acquire::https=1 temurin-17-jdk=17.0.9.0.0+9)

Also, I observed that the 403 is not thrown at the very beginning but after some content is already downloaded (~15 %). I guess that APT opens multiple connections to download packages and a one of those workers got the 403 failing the rest of the download:

root@3d636ca4e8cf:/home/user# ls -la /var/cache/apt/archives/partial/
total 16396
drwx------ 1 _apt root     4096 Nov 22 21:36 .
drwxr-xr-x 1 root root     4096 Nov 22 21:29 ..
-rw-r--r-- 1 root root 16777216 Oct 30 14:12 temurin-17-jdk_17.0.9.0.0+9_amd64.deb
root@3d636ca4e8cf:/home/user# rm /var/cache/apt/archives/partial/temurin-17-jdk_17.0.9.0.0+9_amd64.deb

Here are the outputs of APT with debug options when it failed and when it worked:

Failed with 403 downloading latest version

``` root@3d636ca4e8cf:/home/user# DEBIAN_FRONTEND=noninteractive apt-get install -oDebug::Acquire::https=1 -y --no-install-recommends temurin-17-jdk=17.0.9.0.0+9 Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: adoptium-ca-certificates java-common libp11-kit0 p11-kit p11-kit-modules Suggested packages: default-jre Recommended packages: fonts-dejavu-extra The following NEW packages will be installed: adoptium-ca-certificates java-common p11-kit p11-kit-modules temurin-17-jdk The following packages will be upgraded: libp11-kit0 1 upgraded, 5 newly installed, 0 to remove and 68 not upgraded. Need to get 166 MB/166 MB of archives. After this operation, 333 MB of additional disk space will be used. 17% [Working]GET /artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.9.0.0%2b9_amd64.deb HTTP/1.1 Host: packages.adoptium.net User-Agent: Debian APT-HTTP/1.3 (1.6.12ubuntu0.1) Answer for: https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.9.0.0+9_amd64.deb HTTP/1.1 200 OK Connection: keep-alive Content-Length: 165543916 x-amz-id-2: 6YwnzVTdwz8yhZkAUYs6NO9vF5PnAQQjCm+5XMRsuNOWikFKSXLJrgAvaI32rRIfzNOALiLxyF8= x-amz-request-id: YKK0FBYJVA67M8PS x-amz-replication-status: FAILED Last-Modified: Mon, 30 Oct 2023 14:12:22 GMT ETag: "cc468a88d7235d8911184b421b036031" x-amz-server-side-encryption: AES256 x-amz-version-id: mOcoiHLQ7jeVUun7cgkPTP36Z3b_Hzu8 Content-Type: application/x-debian-package Server: AmazonS3 Cache-Control: public, max-age=15780096.000 Fastly-Restarts: 1 Accept-Ranges: bytes Date: Wed, 22 Nov 2023 21:41:36 GMT Via: 1.1 varnish Age: 3119 X-Served-By: cache-eze2230026-EZE X-Cache: HIT X-Cache-Hits: 1 X-Timer: S1700689297.581823,VS0,VE12 Get:1 https://packages.adoptium.net/artifactory/deb bionic/main amd64 temurin-17-jdk amd64 17.0.9.0.0+9 [166 MB] 24% [1 temurin-17-jdk 13.8 MB/166 MB 8%]GET /artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.9.0.0%2b9_amd64.deb HTTP/1.1 Host: packages.adoptium.net Range: bytes=16777216- If-Range: Mon, 30 Oct 2023 14:12:22 GMT User-Agent: Debian APT-HTTP/1.3 (1.6.12ubuntu0.1) 25% [1 temurin-17-jdk 16.8 MB/166 MB 10%]Answer for: https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.9.0.0+9_amd64.deb HTTP/1.1 403 Forbidden Connection: keep-alive x-amz-request-id: F5S03N8J3B9NJN7T x-amz-id-2: 9R/nDMoJQWyrBQY14fvoph09ODiZOXaWEtVzKtvsngVZrg0FQIIy06hLuGnHn7NXoxnp88rwOyc= Content-Type: application/xml Server: AmazonS3 Fastly-Restarts: 1 Accept-Ranges: bytes Date: Wed, 22 Nov 2023 21:41:38 GMT Via: 1.1 varnish X-Served-By: cache-eze2230058-EZE X-Cache: MISS X-Cache-Hits: 0 X-Timer: S1700689298.102621,VS0,VE749 transfer-encoding: chunked Err:1 https://packages.adoptium.net/artifactory/deb bionic/main amd64 temurin-17-jdk amd64 17.0.9.0.0+9 403 Forbidden [IP: 151.101.219.42 443] E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.9.0.0+9_amd64.deb 403 Forbidden [IP: 151.101.219.42 443] E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? root@3d636ca4e8cf:/home/user# ```

Downloaded latest version some minutes later

``` root@3d636ca4e8cf:/home/user# DEBIAN_FRONTEND=noninteractive apt-get install -oDebug::Acquire::https=1 -y --no-install-recommends temurin-17-jdk=17.0.9.0.0+9 Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: adoptium-ca-certificates java-common p11-kit p11-kit-modules Suggested packages: default-jre Recommended packages: fonts-dejavu-extra The following NEW packages will be installed: adoptium-ca-certificates java-common p11-kit p11-kit-modules temurin-17-jdk 0 upgraded, 5 newly installed, 0 to remove and 68 not upgraded. Need to get 166 MB of archives. After this operation, 332 MB of additional disk space will be used. 0% [Working]GET /artifactory/deb/pool/main/a/adoptium-ca-certificates/adoptium-ca-certificates_1.0.2-1_all.deb HTTP/1.1 Host: packages.adoptium.net User-Agent: Debian APT-HTTP/1.3 (1.6.12ubuntu0.1) Answer for: https://packages.adoptium.net/artifactory/deb/pool/main/a/adoptium-ca-certificates/adoptium-ca-certificates_1.0.2-1_all.deb HTTP/1.1 200 OK Connection: keep-alive Content-Length: 2304 Content-Type: application/x-debian-package X-JFrog-Version: Artifactory/7.71.1 77101900 X-Artifactory-Id: 5da408aece77f41502230d8ecf66e33487a6aab2 X-Artifactory-Node-Id: adoptium-artifactory-primary-0 Last-Modified: Tue, 20 Jun 2023 16:04:14 GMT ETag: 0347ba993198898a773c8ba4084e8022d3dd1bfb X-Checksum-Sha1: 0347ba993198898a773c8ba4084e8022d3dd1bfb X-Checksum-Sha256: f944bcc23005177751212e578437065d6eedbfa57e26b1821b7f12456dcb5413 X-Checksum-Md5: 5db8bf5481fe4ad127e500d9ffc4576f X-Artifactory-Filename: adoptium-ca-certificates_1.0.2-1_all.deb Strict-Transport-Security: max-age=31536000; includeSubDomains X-Request-ID: 3554394ae9c346a390b48a1de9713501 Cache-Control: public, max-age=15780096.000 Accept-Ranges: bytes Date: Wed, 22 Nov 2023 21:50:36 GMT Via: 1.1 varnish Age: 1226858 X-Served-By: cache-eze2230064-EZE X-Cache: HIT X-Cache-Hits: 2 X-Timer: S1700689837.703204,VS0,VE0 Get:1 https://packages.adoptium.net/artifactory/deb bionic/main amd64 adoptium-ca-certificates all 1.0.2-1 [2304 B] GET /artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.9.0.0%2b9_amd64.deb HTTP/1.1 Host: packages.adoptium.net User-Agent: Debian APT-HTTP/1.3 (1.6.12ubuntu0.1) Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 p11-kit-modules amd64 0.23.9-2ubuntu0.1 [234 kB] Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 p11-kit amd64 0.23.9-2ubuntu0.1 [93.2 kB] Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 java-common all 0.68ubuntu1~18.04.1 [14.5 kB] 16% [Waiting for headers]Answer for: https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.9.0.0+9_amd64.deb HTTP/1.1 200 OK Connection: keep-alive Content-Length: 165543916 x-amz-id-2: I31eBDk1re2GAh0BEOv+Nd0qnZVHNhMCAtTxW0InaJczvUInPLeqy1r81ucR0TlgaX2exAZDsEc= x-amz-request-id: CX5K6GGZFPCHNFPD x-amz-replication-status: FAILED Last-Modified: Mon, 30 Oct 2023 14:12:22 GMT ETag: "cc468a88d7235d8911184b421b036031" x-amz-server-side-encryption: AES256 x-amz-version-id: mOcoiHLQ7jeVUun7cgkPTP36Z3b_Hzu8 Content-Type: application/x-debian-package Server: AmazonS3 Cache-Control: public, max-age=15780096.000 Fastly-Restarts: 1 Accept-Ranges: bytes Date: Wed, 22 Nov 2023 21:50:40 GMT Via: 1.1 varnish Age: 2 X-Served-By: cache-eze2230064-EZE X-Cache: MISS X-Cache-Hits: 0 X-Timer: S1700689837.713358,VS0,VE3556 Get:5 https://packages.adoptium.net/artifactory/deb bionic/main amd64 temurin-17-jdk amd64 17.0.9.0.0+9 [166 MB] Fetched 166 MB in 49s (3365 kB/s) debconf: delaying package configuration, since apt-utils is not installed Selecting previously unselected package p11-kit-modules:amd64. (Reading database ... 25417 files and directories currently installed.) Preparing to unpack .../p11-kit-modules_0.23.9-2ubuntu0.1_amd64.deb ... Unpacking p11-kit-modules:amd64 (0.23.9-2ubuntu0.1) ... Selecting previously unselected package p11-kit. Preparing to unpack .../p11-kit_0.23.9-2ubuntu0.1_amd64.deb ... Unpacking p11-kit (0.23.9-2ubuntu0.1) ... Selecting previously unselected package adoptium-ca-certificates. Preparing to unpack .../adoptium-ca-certificates_1.0.2-1_all.deb ... Unpacking adoptium-ca-certificates (1.0.2-1) ... Selecting previously unselected package java-common. Preparing to unpack .../java-common_0.68ubuntu1~18.04.1_all.deb ... Unpacking java-common (0.68ubuntu1~18.04.1) ... Selecting previously unselected package temurin-17-jdk. Preparing to unpack .../temurin-17-jdk_17.0.9.0.0+9_amd64.deb ... Unpacking temurin-17-jdk (17.0.9.0.0+9) ... Setting up java-common (0.68ubuntu1~18.04.1) ... Setting up p11-kit-modules:amd64 (0.23.9-2ubuntu0.1) ... Setting up p11-kit (0.23.9-2ubuntu0.1) ... Setting up adoptium-ca-certificates (1.0.2-1) ... /etc/ssl/certs/adoptium/cacerts successfully populated. Setting up temurin-17-jdk (17.0.9.0.0+9) ... update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jar to provide /usr/bin/jar (jar) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jarsigner to provide /usr/bin/jarsigner (jarsigner) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/java to provide /usr/bin/java (java) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/javac to provide /usr/bin/javac (javac) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/javadoc to provide /usr/bin/javadoc (javadoc) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/javap to provide /usr/bin/javap (javap) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jcmd to provide /usr/bin/jcmd (jcmd) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jconsole to provide /usr/bin/jconsole (jconsole) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jdb to provide /usr/bin/jdb (jdb) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jdeprscan to provide /usr/bin/jdeprscan (jdeprscan) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jdeps to provide /usr/bin/jdeps (jdeps) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jfr to provide /usr/bin/jfr (jfr) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jhsdb to provide /usr/bin/jhsdb (jhsdb) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jimage to provide /usr/bin/jimage (jimage) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jinfo to provide /usr/bin/jinfo (jinfo) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jlink to provide /usr/bin/jlink (jlink) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jmap to provide /usr/bin/jmap (jmap) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jmod to provide /usr/bin/jmod (jmod) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jpackage to provide /usr/bin/jpackage (jpackage) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jps to provide /usr/bin/jps (jps) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jrunscript to provide /usr/bin/jrunscript (jrunscript) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jshell to provide /usr/bin/jshell (jshell) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jstack to provide /usr/bin/jstack (jstack) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jstat to provide /usr/bin/jstat (jstat) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jstatd to provide /usr/bin/jstatd (jstatd) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/keytool to provide /usr/bin/keytool (keytool) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/rmiregistry to provide /usr/bin/rmiregistry (rmiregistry) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/serialver to provide /usr/bin/serialver (serialver) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/lib/jexec to provide /usr/bin/jexec (jexec) in auto mode update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/lib/jspawnhelper to provide /usr/bin/jspawnhelper (jspawnhelper) in auto mode Processing triggers for man-db (2.8.3-2ubuntu0.1) ... root@3d636ca4e8cf:/home/user# ```

I suggest looking to x-amz-replication-status (AWS doc, and how it can be monitored), the X-Cache and Age headers.

Just for reference, these are other reported issues that I come across while researching the problem.... it seems that the issue is not new:

• https://github.com/adoptium/adoptium-support/issues/932
• https://github.com/adoptium/adoptium-support/issues/632
• https://github.com/adoptium/adoptium-support/issues/554

[saamorim]

We too are facing this issue everyday, and it is not clear why.... Only during some moments we can actually install the package... is there any alternative for https://packages.adoptium.net/artifactory/deb that it is more reliable? or any known mitigation for this?

Thank you

[Konig-Corey]

I can confirm that we are as well seeing this intermittent issue. Certs do not appear to be the issue. Bumping this thread. 👍

[hpernu]

Seeing similar errors with debian package manager (apt) as well several times per day but not always.

[DevOpsAzurance]

Seem to be getting this error here still:

azure-arm.build_image: The following NEW packages will be installed: azure-arm.build_image: temurin-21-jdk azure-arm.build_image: 0 upgraded, 1 newly installed, 0 to remove and 19 not upgraded. azure-arm.build_image: Need to get 177 MB of archives. azure-arm.build_image: After this operation, 360 MB of additional disk space will be used. azure-arm.build_image: Err:1 https://packages.adoptium.net/artifactory/deb jammy/main amd64 temurin-21-jdk amd64 21.0.1.0.0+12 azure-arm.build_image: 403 Forbidden [IP: 146.75.39.42 443] ==> azure-arm.build_image: E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-21/temurin-21-jdk_21.0.1.0.0%2b12_amd64.deb 403 Forbidden [IP: 146.75.39.42 443] ==> azure-arm.build_image: E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? ==> azure-arm.build_image: cat: /usr/lib/jvm/temurin-21-jdk-amd64/release: No such file or directory azure-arm.build_image: Java 21 Toolcache Version Path: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk//usr/lib/jvm/temurin-21-jdk-amd64/bin/java: azure-arm.build_image: Setting up JAVA_HOME_21_X64 variable to /usr/lib/jvm/temurin-21-jdk-amd64 azure-arm.build_image: JAVA_HOME_21_X64=/usr/lib/jvm/temurin-21-jdk-amd64

The JAVA_HOME_21_X64 is being set and thus this fails:

azure-arm.build_image: [-] Java 21 33ms (30ms|3ms) azure-arm.build_image: Command '"/usr/lib/jvm/temurin-21-jdk-amd64/bin/java" -version' has finished with exit code azure-arm.build_image: /usr/bin/bash: line 1: /usr/lib/jvm/temurin-21-jdk-amd64/bin/java: No such file or directory azure-arm.build_image: at ""$javaPath" -version" | Should -ReturnZeroExitCode, /imagegeneration/tests/Java.Tests.ps1:43 azure-arm.build_image: at <ScriptBlock>, /imagegeneration/tests/Java.Tests.ps1:43

I was having other issues on my Windows build of my runner-images build, which I attributed to Defender. Been adding the 'ExcludeMdeAutoProvisioning' tag to disable provisioning on both the Windows and Linux builds, but this seems to not be an issue on the Azure side, but the adoptium endpoint. I had a similar issue with VS extensions and the UI behavior doing a redirect that CLI was not getting properly, which resulted in me adding logic that sets redirect to "0" then gets the resulting 'Location' header to make the call correctly. I was then getting a CDN error due to limits on calls to the service in a given time. This came back with a 'Retry-After' header, that I used to retry the call after the time I was given. This worked for me. Not sure if something similar is needed here. This will be dependent on the CDN giving a response with a header that can be used.

[karianna]

Hey folks, once we've worked through the latest PSU release we'll be getting back to investigating this. Just wanted to drop a note that this isn't being ignored :-)

[vamshisiram]

Hi @karianna I am facing an issue with devops builds and was notified that this issue is blocking me.

https://github.com/actions/runner-images/issues/9220 this is the issue i am facing, Let us know if there is any workaround.

azure-arm.build_image: Get:1 https://packages.adoptium.net/artifactory/deb focal/main amd64 temurin-21-jdk amd64 21.0.1.0.0+12 [177 MB] azure-arm.build_image: Err:1 https://packages.adoptium.net/artifactory/deb focal/main amd64 temurin-21-jdk amd64 21.0.1.0.0+12 azure-arm.build_image: 403 Forbidden [IP: 146.75.107.42 443] ==> azure-arm.build_image: E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-21/temurin-21-jdk_21.0.1.0.0+12_amd64.deb 403 Forbidden [IP: 146.75.107.42 443] ==> azure-arm.build_image: E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

[karianna]

It likely is, once we've completed this PSU release cycle folks will take a fresh look.

[vamshisiram]

@karianna when is that release cycle? Just so i can check back and keep informing my company internally on when we can expect stuff to work again.

[karianna]

See the banner on our website for the updates

[classicrocker883]

this issue still remains... using Debian- bookworm. temurin-17-jdk worked fine, except -jre gives this error

Err:1 https://packages.adoptium.net/artifactory/deb bookworm/main amd64 temurin-17-jre amd64 17.0.10.0.0+7
  400  Bad Request [IP: 151.101.119.42 443]
E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-${VERSION}/temurin-17-jre_17.0.10.0.0%2b7_amd64.deb  400  Bad Request [IP: 151.101.119.42 443]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

this is the URL from the website actually works if it were replaced instead.... https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-$%7BVERSION%7D/temurin-17-jre_17.0.10.0.0%2b7_amd64.deb

[jappenzesr]

Also affected, when trying to build runner-images, blocking new releases of the same image. Thanks for looking into it.

[ijaouani]

I was reproducing this issue on last Thursday but not anymore today. Seems it has been fixed.

[panther7]

Still issue...

Do you want to continue? [Y/n]
Err:1 https://packages.adoptium.net/artifactory/deb bookworm/main amd64 temurin-17-jdk amd64 17.0.10.0.0+7
  403  Forbidden [IP: 199.232.19.42 443]
E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.10.0.0%2b7_amd64.deb  403  Forbidden [IP: 199.232.19.42 443]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

[classicrocker883]

one workaround is to simply download the .deb from the website, you may need to change the url to get it to work, but just use dpkg -i

[dustingrady]

I was able to resolve this issue by using the Mac .dmg version instead of the Linux version

On Monday, January 29th, 2024 at 13:14, Andrew @.***> wrote:

one workaround is to simply download the .deb from the website, you may need to change the url to get it to work, but just use dpkg - i

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[skalderqm]

received 403 and 503 errors as well as Hash Sum mismatch yesterday:

E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.10.0.0%2b7_amd64.deb 503 Response object too large [IP: 151.101.3.42 443]

E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.10.0.0%2b7_amd64.deb 403 Forbidden [IP: 151.101.67.42 443]

E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.10.0.0%2b7_amd64.deb 403 Forbidden [IP: 151.101.131.42 443]

E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.10.0.0%2b7_amd64.deb 503 Response object too large [IP: 151.101.195.42 443]

E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.10.0.0%2b7_amd64.deb Hash Sum mismatch

[mhever]

Still facing this issue

==> azure-arm.build_image: <- https:102%20Status%0aMessage:%20Waiting%20for%20headers%0aURI:%20https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-11/temurin-11-jdk_11.0.22.0.0%252b7_amd64.deb ==> azure-arm.build_image: Answer for: https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-11/temurin-11-jdk_11.0.22.0.0%2b7_amd64.deb ==> azure-arm.build_image: HTTP/1.1 403 Forbidden ==> azure-arm.build_image: Connection: keep-alive ==> azure-arm.build_image: x-amz-request-id: K1339498VN86JVVG ==> azure-arm.build_image: x-amz-id-2: U6hqQj+0Z2Fj/WHkuYEcqXzfuP70mOAB041ddaqxQp8nXxk4lY8UdsMAa/Euf1h+g0gPpHF0Q7Y= ==> azure-arm.build_image: Content-Type: application/xml ==> azure-arm.build_image: Server: AmazonS3 ==> azure-arm.build_image: Fastly-Restarts: 1 ==> azure-arm.build_image: Accept-Ranges: bytes ==> azure-arm.build_image: Date: Wed, 07 Feb 2024 14:14:41 GMT ==> azure-arm.build_image: Via: 1.1 varnish ==> azure-arm.build_image: X-Served-By: cache-ams21021-AMS ==> azure-arm.build_image: X-Cache: MISS ==> azure-arm.build_image: X-Cache-Hits: 0 ==> azure-arm.build_image: X-Timer: S1707315282.777892,VS0,VE158 ==> azure-arm.build_image: transfer-encoding: chunked ==> azure-arm.build_image: ==> azure-arm.build_image: <- https:400%20URI%20Failure%0aFailReason:%20HttpError403%0aMessage:%20403%20%20Forbidden%20[IP:%20151.101.39.42%20443]%0aURI:%20https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-11/temurin-11-jdk_11.0.22.0.0%252b7_amd64.deb ==> azure-arm.build_image: E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-11/temurin-11-jdk_11.0.22.0.0%2b7_amd64.deb Error reading from server. Remote end closed connection [IP: 151.101.39.42 443] ==> azure-arm.build_image: E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? ==> azure-arm.build_image: cat: /usr/lib/jvm/temurin-11-jdk-amd64/release: No such file or directory

[ericlayton]

This error is continuing to occur for at least Temurin 11 and 21 packages.

10:22:54 Reading package lists... 10:22:54 Building dependency tree... 10:22:54 Reading state information... 10:22:54 The following NEW packages will be installed: 10:22:54 temurin-21-jdk 10:22:54 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. 10:22:54 Need to get 178 MB of archives. 10:22:54 After this operation, 362 MB of additional disk space will be used. 10:22:54 Get:1 https://packages.adoptium.net/artifactory/deb focal/main amd64 temurin-21-jdk amd64 21.0.2.0.0+13-2 [178 MB] 10:22:55 Err:1 https://packages.adoptium.net/artifactory/deb focal/main amd64 temurin-21-jdk amd64 21.0.2.0.0+13-2 10:22:55 403 Forbidden [IP: 146.75.35.42 443] 10:22:55 E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-21/temurin-21-jdk_21.0.2.0.0+13-2_amd64.deb 403 Forbidden [IP: 146.75.35.42 443] 10:22:55 E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

11:22:06 Reading package lists... 11:22:06 Building dependency tree... 11:22:07 Reading state information... 11:22:07 The following NEW packages will be installed: 11:22:07 temurin-11-jdk 11:22:07 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. 11:22:07 Need to get 168 MB of archives. 11:22:07 After this operation, 325 MB of additional disk space will be used. 11:22:07 Err:1 https://packages.adoptium.net/artifactory/deb focal/main amd64 temurin-11-jdk amd64 11.0.22.0.0+7 11:22:07 403 Forbidden [IP: 146.75.35.42 443] 11:22:07 E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-11/temurin-11-jdk_11.0.22.0.0+7_amd64.deb 403 Forbidden [IP: 146.75.35.42 443] 11:22:07 E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

[jbonofre]

We are facing this issue (a bunch of 403 errors trying to bootstrap, same as @ericlayton shared). @gdams any update in this one ? Maybe @mbarbero can help ?

[fabio-blanco]

Forbidden error happening with an Ubuntu 20.04 while running sudo apt upgrade:

Lendo listas de pacotes... Pronto
Construindo árvore de dependências       
Lendo informação de estado... Pronto
Calculando atualização... Pronto
Obtenha mais atualizações de segurança através do Ubuntu Pro com 'esm-apps' habilitado:
  libavformat58 libmagick++-6.q16-8 libavfilter7 libimage-magick-perl
  libmagickcore-6.q16-6-extra libimage-magick-q16-perl imagemagick
  libswresample3 libgegl-0.4-0 libzmq5 libmagickwand-6.q16-6 libgegl-common
  libcgraph6 libpostproc55 liblab-gamut1 imagemagick-6.q16 libavcodec58
  libcdt5 libavutil56 libpathplan4 libswscale5 libgvpr2 libopenexr24
  libsdl2-2.0-0 libmysofa1 libmagickcore-6.q16-6 libgvc6 graphviz
  imagemagick-6-common
Saiba mais sobre o Ubuntu Pro em https://ubuntu.com/pro
Os pacotes a seguir serão atualizados:
  temurin-21-jdk
1 pacotes atualizados, 0 pacotes novos instalados, 0 a serem removidos e 0 não atualizados.
É preciso baixar 178 MB de arquivos.
Depois desta operação, 738 kB adicionais de espaço em disco serão usados.
Você quer continuar? [S/n] 
Err:1 https://packages.adoptium.net/artifactory/deb focal/main amd64 temurin-21-jdk amd64 21.0.2.0.0+13-2
  403  Forbidden [IP: 151.101.3.42 443]
E: Falhou ao buscar https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-21/temurin-21-jdk_21.0.2.0.0+13-2_amd64.deb  403  Forbidden [IP: 151.101.3.42 443]
E: Impossível buscar alguns arquivos, talvez executar apt-get update ou tentar com --fix-missing?

[jbonofre]

We are still facing this issue (randomly but pretty often). It's basically the same as @mhever. I'm trying a bunch of curl (with traceroute) to have some clues.

It would be great to have an update from Adoptium committers and/or Eclipse infra.

[jerboaa]

It would be great to have an update from Adoptium committers and/or Eclipse infra.

I'm one of the Adoptium PMC members. We are aware of the issue and keep monitoring the situation. For example, we discussed this issue again on the PMC meeting of March 20'th, 2024. Unfortunately, we are unaware of a proper reproducer and given the volume of hits we get to the repository the failure cases are fairly infrequent (relatively speaking, considering the scale). In the absence of a reproducer, there is unfortunately only a limited amount that we can do on our end. We keep monitoring the situation with an eye on the failure rate, however.

Having said that, if somebody ends up figuring out what exactly is needed to reliably reproduce the issue we can work on a more permanent solution. As of yet, there is little we can do. Thanks for your understanding!

[jbonofre]

@jerboaa Thanks for your feedback ! I agree, I'm also struggling to reproduce the behavior on my machine, it seems way more frequent on our cloud infra. Let me try to find a reproducer.

Thanks again !

[theadzik]

Got the error today:

wget -qO - https://packages.adoptium.net/artifactory/api/gpg/key/public | gpg --dearmor > /usr/share/keyrings/adoptium.gpg
echo "deb [signed-by=/usr/share/keyrings/adoptium.gpg] https://packages.adoptium.net/artifactory/deb/ $(lsb_release -cs) main" > /etc/apt/sources.list.d/adoptium.list
apt-get -q update
apt-get -q install temurin-17-jdk=\*

#13 194.0 Reading package lists...
#13 194.7 Building dependency tree...
#13 194.8 Reading state information...
#13 195.0 The following NEW packages will be installed:
#13 195.0   temurin-17-jdk
#13 195.4 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
#13 195.4 Need to get 166 MB of archives.
#13 195.4 After this operation, 331 MB of additional disk space will be used.
#13 195.4 Get:1 https://packages.adoptium.net/artifactory/deb jammy/main amd64 temurin-17-jdk amd64 17.0.11.0.0+9 [166 MB]
#13 203.2 Ign:1 https://packages.adoptium.net/artifactory/deb jammy/main amd64 temurin-17-jdk amd64 17.0.11.0.0+9
#13 205.1 Err:1 https://packages.adoptium.net/artifactory/deb jammy/main amd64 temurin-17-jdk amd64 17.0.11.0.0+9
#13 205.1   Error reading from server. Remote end closed connection [IP: 146.75.123.42 443]
#13 205.1 E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.11.0.0%2b9_amd64.deb  Error reading from server. Remote end closed connection [IP: 146.75.123.42 443]
#13 205.1 E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

It worked 2 hours ago, now I can't get through after multiple tries

EDIT: It started working after an hour

[poiema72]

I'm also experiencing similar issue today. (Experienced similar issue yesterday, As commented above, sometimes it worked yesterday.) However, again, happending today.

32.76 Err:6 https://packages.adoptium.net/artifactory/deb bullseye/main amd64 temurin-8-jdk amd64 8.0.412.0.0+8
32.76   Connection timed out [IP: 146.75.51.42 443]
32.77 E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-8/temurin-8-jdk_8.0.412.0.0%2b8_amd64.deb  Connection timed out [IP: 146.75.51.42 443]
32.77 E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
32.77 Fetched 7195 kB in 31s (233 kB/s)

EDIT: Same as above. It started working now.

[ainoelina]

Getting the same issue when trying to use packer image for Ubuntu 20.04

azure-arm.build_image: Get:1 https://packages.adoptium.net/artifactory/deb focal/main amd64 temurin-11-jdk amd64 11.0.23.0.0+9 [168 MB]
azure-arm.build_image: Err:1 https://packages.adoptium.net/artifactory/deb focal/main amd64 temurin-11-jdk amd64 11.0.23.0.0+9
azure-arm.build_image:   403  Forbidden [IP: 151.101.39.42 443]
==> azure-arm.build_image: E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-11/temurin-11-jdk_11.0.23.0.0+9_amd64.deb  403  Forbidden [IP: 151.101.39.42 443]
==> azure-arm.build_image: E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?