search

axel-download-accelerator / axel

Lightweight CLI download accelerator
GNU General Public License v2.0
2.85k stars 258 forks source link

ERROR 403: Forbidden on specific sites #383

Closed git-bruh closed 1 year ago

git-bruh commented 1 year ago

https://www.netfilter.org/projects/iptables/files/iptables-1.8.8.tar.bz2

openssl 3.0.5

curl:

*   Trying 92.243.18.11:443...
* Connected to www.netfilter.org (92.243.18.11) port 443 (#0)
* ALPN: offers http/1.1
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN: server did not agree on a protocol. Uses default.
* Server certificate:
*  subject: CN=iptables.org
*  start date: Sep  8 22:01:02 2022 GMT
*  expire date: Dec  7 22:01:01 2022 GMT
*  subjectAltName: host "www.netfilter.org" matched cert's "www.netfilter.org"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET /projects/iptables/files/iptables-1.8.8.tar.bz2 HTTP/1.1
> Host: www.netfilter.org
> User-Agent: curl/7.85.0
> Accept: */*
> 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Fri, 09 Sep 2022 10:32:18 GMT
< Server: Apache
< Last-Modified: Fri, 13 May 2022 13:49:59 GMT
< ETag: "b65e9-5dee4f34827d8"
< Accept-Ranges: bytes
< Content-Length: 746985
< Content-Type: application/x-bzip2
< 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
Warning: Binary output can mess up your terminal. Use "--output -" to tell 
Warning: curl to output it to your terminal anyway, or consider "--output 
Warning: <FILE>" to save to a file.
* Failure writing output to destination
* Closing connection 0
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Unknown (21):
* TLSv1.2 (OUT), TLS alert, close notify (256):

axel:

axel https://www.netfilter.org/projects/iptables/files/iptables-1.8.8.tar.bz2
Initializing download: https://www.netfilter.org/projects/iptables/files/iptables-1.8.8.tar.bz2
ERROR 403: Forbidden.
axel --user-agent='curl/7.85.0' https://www.netfilter.org/projects/iptables/files/iptables-1.8.8.tar.bz2
Initializing download: https://www.netfilter.org/projects/iptables/files/iptables-1.8.8.tar.bz2
ERROR 403: Forbidden.
ismaell commented 1 year ago

It's possibly due to Axel using HTTP/1.0, can you confirm that?

git-bruh commented 1 year ago

Did you mean to use curl with HTTP/1 and try? I patched it to use http1.0 and it still works

λ ./src/curl -Lv https://www.netfilter.org/projects/iptables/files/iptab
les-1.8.8.tar.bz2
*   Trying 92.243.18.11:443...
* Connected to www.netfilter.org (92.243.18.11) port 443 (#0)
* ALPN: offers http/1.1
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN: server did not agree on a protocol. Uses default.
* Server certificate:
*  subject: CN=iptables.org
*  start date: Sep  8 22:01:02 2022 GMT
*  expire date: Dec  7 22:01:01 2022 GMT
*  subjectAltName: host "www.netfilter.org" matched cert's "www.netfilter.org"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET /projects/iptables/files/iptables-1.8.8.tar.bz2 HTTP/1.0
> Host: www.netfilter.org
> User-Agent: curl/7.85.0
> Accept: */*
> 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Fri, 09 Sep 2022 12:39:21 GMT
< Server: Apache
< Last-Modified: Fri, 13 May 2022 13:49:59 GMT
< ETag: "b65e9-5dee4f34827d8"
< Accept-Ranges: bytes
< Content-Length: 746985
< Connection: close
< Content-Type: application/x-bzip2
< 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
Warning: Binary output can mess up your terminal. Use "--output -" to tell 
Warning: curl to output it to your terminal anyway, or consider "--output 
Warning: <FILE>" to save to a file.
* Failure writing output to destination
* Closing connection 0
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Unknown (21):
* TLSv1.2 (OUT), TLS alert, close notify (256):

Changing HTTP/1.0 to HTTP/1.1 axel src/http.c doesn't make any difference either

ismaell commented 1 year ago

The server doesn't like/want range requests:

$ curl -r 0-99 https://www.netfilter.org/projects/iptables/files/iptables-1.8.8.tar.bz2
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /projects/iptables/files/iptables-1.8.8.tar.bz2
on this server.</p>
</body></html>

I don't think we can do much about that, it's a server-side problem, please report it to the project.

403 Forbidden is the wrong error code for the server to return in this case, it should be 416 Range Not Satisfiable.

git-bruh commented 1 year ago

Alright, thanks for looking into it!