search

kitabisa / mubeng

An incredibly fast proxy checker & IP rotator with ease.
Apache License 2.0
1.59k stars 187 forks source link

[BUG] http->https redirects are causing redirect loop and "stopped after 10 redirects" error message #230

Closed yudelevi closed 3 weeks ago

yudelevi commented 7 months ago

Describe the bug

I was trying to troubleshoot why I'm getting "server gave HTTP response to HTTPS client" and "stopped after 10 redirects" in abnormal amounts, and I believe I stumbled upon a bug.

In error log: 2024/02/24 03:26:59 [DEBU] ▶ 10.200.1.20:37806 GET http://www.bunch.ca/about 2024/02/24 03:27:07 [ERRO] ▶ 10.200.1.20:37806 Get "https://www.bunch.ca/about": stopped after 10 redirects

To narrow it down, I've run the following:

  1. curl -v -i -L -x "mubeng_host:port" https://www.bunch.ca/about
  2. curl -v -i -L -x "mubeng_host:port" http://www.bunch.ca/about
  3. curl -v -i -L -x "proxy_host:port" https://www.bunch.ca/about
  4. curl -v -i -L -x "proxy_host:port" http://www.bunch.ca/about

Cases 1, 3, and 4 worked, while two resulted in :

HTTP/1.1 502 Bad Gateway
Content-Type: text/plain
Date: Sat, 24 Feb 2024 03:27:07 GMT
Content-Length: 18

Proxy server error

and the error in the error log

Environment (please complete the following information):

Single HTTP proxy in this test, running as: /usr/bin/mubeng -f /etc/default/proxies -a :3153 -A -g 20 -m random -w -v

Additional context

Direct to HTTPS:

> X-Forwarded-Proto: http

< HTTP/1.1 200 OK
< Access-Control-Allow-Credentials: true
< Access-Control-Allow-Origin: *
< Content-Length: 0
< Content-Type: text/html; charset=utf-8
< Date: Sat, 24 Feb 2024 03:26:29 GMT
< Server: gunicorn/19.9.0

to HTTP

* Request to http://www.bunch.ca/about
* Request from 10.200.1.20:37806
* Skipping TLS verification: connection is susceptible to man-in-the-middle attacks.
> GET /about HTTP/1.1
> Host: www.bunch.ca
> Accept: */*
> User-Agent: curl/7.76.1
> X-Forwarded-For: ******
> X-Forwarded-Proto: http

< HTTP/1.1 308 Permanent Redirect
< Content-Length: 62
< Content-Type: text/html; charset=utf-8
< Date: Sat, 24 Feb 2024 03:27:57 GMT
< Location: https://www.bunch.ca/about
< Server: openresty
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Backend: varnish_ssl

* Request to https://www.bunch.ca/about
* Skipping TLS verification: connection is susceptible to man-in-the-middle attacks.
> GET /about 
> Host: www.bunch.ca
> Accept: */*
> Referer: http://www.bunch.ca/about
> User-Agent: curl/7.76.1
> X-Forwarded-For: ******
> X-Forwarded-Proto: http

< HTTP/1.1 308 Permanent Redirect
< Content-Length: 62
< Content-Type: text/html; charset=utf-8
< Date: Sat, 24 Feb 2024 03:25:39 GMT
< Location: https://www.bunch.ca/about
< Server: openresty
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Backend: varnish_ssl

* Request to https://www.bunch.ca/about
* Skipping TLS verification: connection is susceptible to man-in-the-middle attacks.
> GET /about 
> Host: www.bunch.ca
> Accept: */*
> Referer: https://www.bunch.ca/about
> User-Agent: curl/7.76.1
> X-Forwarded-For: ******
> X-Forwarded-Proto: http

< HTTP/1.1 308 Permanent Redirect
< Content-Length: 62
< Content-Type: text/html; charset=utf-8
< Date: Sat, 24 Feb 2024 03:31:11 GMT
< Location: https://www.bunch.ca/about
< Server: openresty
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Backend: varnish_ssl

* Request to https://www.bunch.ca/about
* Skipping TLS verification: connection is susceptible to man-in-the-middle attacks.
> GET /about 
> Host: www.bunch.ca
> Accept: */*
> Referer: https://www.bunch.ca/about
> User-Agent: curl/7.76.1
> X-Forwarded-For: ******
> X-Forwarded-Proto: http

< HTTP/1.1 308 Permanent Redirect
< Content-Length: 62
< Content-Type: text/html; charset=utf-8
< Date: Sat, 24 Feb 2024 03:30:56 GMT
< Location: https://www.bunch.ca/about
< Server: openresty
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Backend: varnish_ssl

* Request to https://www.bunch.ca/about
* Skipping TLS verification: connection is susceptible to man-in-the-middle attacks.
> GET /about 
> Host: www.bunch.ca
> Accept: */*
> Referer: https://www.bunch.ca/about
> User-Agent: curl/7.76.1
> X-Forwarded-For: ******
> X-Forwarded-Proto: http

< HTTP/1.1 308 Permanent Redirect
< Content-Length: 62
< Content-Type: text/html; charset=utf-8
< Date: Sat, 24 Feb 2024 03:26:06 GMT
< Location: https://www.bunch.ca/about
< Server: openresty
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Backend: varnish_ssl

* Request to https://www.bunch.ca/about
* Skipping TLS verification: connection is susceptible to man-in-the-middle attacks.
> GET /about 
> Host: www.bunch.ca
> Accept: */*
> Referer: https://www.bunch.ca/about
> User-Agent: curl/7.76.1
> X-Forwarded-For: ******
> X-Forwarded-Proto: http

< HTTP/1.1 308 Permanent Redirect
< Content-Length: 62
< Content-Type: text/html; charset=utf-8
< Date: Sat, 24 Feb 2024 03:29:21 GMT
< Location: https://www.bunch.ca/about
< Server: openresty
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Backend: varnish_ssl

* Request to https://www.bunch.ca/about
* Skipping TLS verification: connection is susceptible to man-in-the-middle attacks.
> GET /about 
> Host: www.bunch.ca
> Accept: */*
> Referer: https://www.bunch.ca/about
> User-Agent: curl/7.76.1
> X-Forwarded-For: ******
> X-Forwarded-Proto: http

< HTTP/1.1 308 Permanent Redirect
< Content-Length: 62
< Content-Type: text/html; charset=utf-8
< Date: Sat, 24 Feb 2024 03:30:57 GMT
< Location: https://www.bunch.ca/about
< Server: openresty
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Backend: varnish_ssl

* Request to https://www.bunch.ca/about
* Skipping TLS verification: connection is susceptible to man-in-the-middle attacks.
> GET /about 
> Host: www.bunch.ca
> Accept: */*
> Referer: https://www.bunch.ca/about
> User-Agent: curl/7.76.1
> X-Forwarded-For: ******
> X-Forwarded-Proto: http

< HTTP/1.1 308 Permanent Redirect
< Content-Length: 62
< Content-Type: text/html; charset=utf-8
< Date: Sat, 24 Feb 2024 03:23:27 GMT
< Location: https://www.bunch.ca/about
< Server: openresty
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Backend: varnish_ssl

* Request to https://www.bunch.ca/about
* Skipping TLS verification: connection is susceptible to man-in-the-middle attacks.
> GET /about 
> Host: www.bunch.ca
> Accept: */*
> Referer: https://www.bunch.ca/about
> User-Agent: curl/7.76.1
> X-Forwarded-For: ******
> X-Forwarded-Proto: http

< HTTP/1.1 308 Permanent Redirect
< Content-Length: 62
< Content-Type: text/html; charset=utf-8
< Date: Sat, 24 Feb 2024 03:28:01 GMT
< Location: https://www.bunch.ca/about
< Server: openresty
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Backend: varnish_ssl

* Request to https://www.bunch.ca/about
* Skipping TLS verification: connection is susceptible to man-in-the-middle attacks.
> GET /about 
> Host: www.bunch.ca
> Accept: */*
> Referer: https://www.bunch.ca/about
> User-Agent: curl/7.76.1
> X-Forwarded-For: ******
> X-Forwarded-Proto: http

< HTTP/1.1 308 Permanent Redirect
< Content-Length: 62
< Content-Type: text/html; charset=utf-8
< Date: Sat, 24 Feb 2024 03:30:10 GMT
< Location: https://www.bunch.ca/about
< Server: openresty
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Backend: varnish_ssl