www.php.net blocks "fetch libfetch/2.0" user-agent

[CyberCr33p]

Description

This is what I get when I try to upgrade PHP from 7.4.26 to 7.4.27 using FreeBSD ports:

=> php-7.4.27.tar.xz doesn't seem to exist in /usr/ports/distfiles//. => Attempting to fetch https://www.php.net/distributions/php-7.4.27.tar.xz fetch: https://www.php.net/distributions/php-7.4.27.tar.xz: Service Unavailable => Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/php-7.4.27.tar.xz fetch: http://distcache.FreeBSD.org/ports-distfiles/php-7.4.27.tar.xz: Not Found => Couldn't fetch it - please try to retrieve this => port manually into /usr/ports/distfiles// and try again.

---

FreeBSD ports uses "fetch" to download https://www.php.net/distributions/php-7.4.27.tar.xz but looks like www.php.net web-server blocks user agent "fetch libfetch/2.0" and returns "Service Unavailable".

Can you whitelist this user agent?

PHP Version

PHP 7.4

Operating System

FreeBSD

[cmb69]

curl -A "fetch libfetch/2.0" --output php-7.4.27.tar.xz https://www.php.net/distributions/php-7.4.27.tar.xz works for me. What am I missing?

[cmb69]

Is this possibly certificate related?

[dsh2dsh]

I can confirm:

❯ fetch https://www.php.net/distributions/php-7.4.27.tar.xz
fetch: https://www.php.net/distributions/php-7.4.27.tar.xz: Service Unavailable

[olegsidorkin]

wget https://www.php.net/distributions/php-7.4.27.tar.xz - does work fetch -vvvv https://www.php.net/distributions/php-7.4.27.tar.xz - doesn't. It successfully establishes connection and then fails:

Security CheckOur systems have detected unusual traffic from your computer network. This page checks to see if it is really you sending the requests and not a bot.Please click here to verify that you are not a bot. Browser Working Myra Working Host WorkingWhat happened?You ran into a security check to verify the validity of your request.What can I do?If you are a visitor of this website:You must confirm that you are human.If you are the owner of this website:Please check your security settings.

Headers are: `resolving server address: www.php.net:443 SSL options: 82004854 Peer verification enabled Using CA cert file: /usr/local/etc/ssl/cert.pem Verify hostname TLSv1.2 connection established using ECDHE-RSA-AES128-GCM-SHA256 Certificate subject: /CN=*.php.net Certificate issuer: /C=PL/O=Unizeto Technologies S.A./OU=Certum Certification Authority/CN=Certum Domain Validation CA SHA2 requesting https://www.php.net/distributions/php-7.4.27.tar.xz

GET /distributions/php-7.4.27.tar.xz HTTP/1.1 Host: www.php.net Accept: / User-Agent: fetch libfetch/2.0 Connection: close

<<< HTTP/1.1 503 Service Temporarily Unavailable <<< Server: myracloud <<< Date: Sun, 26 Dec 2021 16:05:06 GMT <<< Content-Type: text/html <<< Transfer-Encoding: chunked <<< Connection: close <<< cache-control: no-cache, no-store, max-age=0 <<< X-Frame-Options: SAMEORIGIN <<< X-Content-Type-Options: nosniff <<< X-XSS-Protection: 1; mode=block`

[cmb69]

Ah, so this would be a systems issue. Maybe @saschaschumann can have a look?

[5u623l20]

I have tried changing user agent of fetch too with no results: fetch -vvvvv --user-agent="curl/7.80.0" https://www.php.net/distributions/php-8.1.1.tar.xz scheme: "https" user: "" password: "" host: "www.php.net" port: "0" document: "/distributions/php-8.1.1.tar.xz" ---> www.php.net:443 resolving server address: www.php.net:443 SSL options: 82004854 Peer verification enabled Using CA cert file: /usr/local/etc/ssl/cert.pem Verify hostname TLSv1.2 connection established using ECDHE-RSA-AES128-GCM-SHA256 Certificate subject: /CN=*.php.net Certificate issuer: /C=PL/O=Unizeto Technologies S.A./OU=Certum Certification Authority/CN=Certum Domain Validation CA SHA2 requesting https://www.php.net/distributions/php-8.1.1.tar.xz

GET /distributions/php-8.1.1.tar.xz HTTP/1.1 Host: www.php.net Accept: / User-Agent: curl/7.80.0 Connection: close

<<< HTTP/1.1 503 Service Temporarily Unavailable <<< Server: myracloud <<< Date: Sun, 26 Dec 2021 16:30:08 GMT <<< Content-Type: text/html <<< Transfer-Encoding: chunked <<< Connection: close <<< cache-control: no-cache, no-store, max-age=0 <<< X-Frame-Options: SAMEORIGIN <<< X-Content-Type-Options: nosniff <<< X-XSS-Protection: 1; mode=block <<< offset 0, length -1, size -1, clength -1 <<< 7738

[5u623l20]

In case someone is already investigating I believe the problem is with http1.1 vs http2. In case we try curl with --http1.1 it also fails with the same errors.